2024-09-21T15:19:24.909 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T15:19:24.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T15:19:25.005 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909985' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4623 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 80/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVx/pr2e3QZWtwdmhJBT4lNtEZOqZ+0MFN3M773+9qypXkCpkcrNMfStw7xxB2MMmljwS/opx1YSiqLPlRtIKc= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHocvVSLooJ9vSsVNgY50udOabyuZbBbmcdmJUJkjjUxzJcqH57RE6CfjG13QxU98Emu4yJbNwQEDcivIp8AqL0= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_22:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T15:19:25.006 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa; will attempt to use it 2024-09-21T15:19:25.006 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa/tasks 2024-09-21T15:19:25.006 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T15:19:25.008 INFO:teuthology.task.internal:Checking packages... 2024-09-21T15:19:25.025 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-21T15:19:25.025 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T15:19:25.025 INFO:teuthology.packaging:ref: None 2024-09-21T15:19:25.025 INFO:teuthology.packaging:tag: None 2024-09-21T15:19:25.025 INFO:teuthology.packaging:branch: reef 2024-09-21T15:19:25.025 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T15:19:25.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-21T15:19:25.248 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-871-g6a1bdf85-1jammy 2024-09-21T15:19:25.249 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T15:19:25.258 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T15:19:25.258 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T15:19:25.271 INFO:teuthology.task.internal:Saving configuration 2024-09-21T15:19:25.282 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T15:19:25.293 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T15:19:25.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 15:13:45.194713', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICkq43bbKrcFxC1S5b1RBx5PhhWzyf0P5dckpXMvwT44'} 2024-09-21T15:19:25.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 15:13:45.193282', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE91fgJ3GdfdGRTvhIGqJmX+xG0r+NZO4CHL0BrgdbwE'} 2024-09-21T15:19:25.339 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T15:19:25.348 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-21T15:19:25.348 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-21T15:19:25.348 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T15:19:25.412 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd2e0d93a30>, signals=[15]) 2024-09-21T15:19:25.412 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T15:19:25.420 INFO:teuthology.task.internal:Opening connections... 2024-09-21T15:19:25.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:19:25.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:19:25.498 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:19:25.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:19:25.579 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T15:19:25.588 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-21T15:19:25.593 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-21T15:19:25.594 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Ubuntu" 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="22.04" 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_CODENAME=jammy 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:ID=ubuntu 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE=debian 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T15:19:25.641 INFO:teuthology.orchestra.run.smithi107.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T15:19:25.642 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T15:19:25.642 INFO:teuthology.orchestra.run.smithi107.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T15:19:25.642 INFO:teuthology.orchestra.run.smithi107.stdout:UBUNTU_CODENAME=jammy 2024-09-21T15:19:25.642 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-21T15:19:25.668 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-09-21T15:19:25.674 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-09-21T15:19:25.674 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04" 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-21T15:19:25.721 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-21T15:19:25.722 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-21T15:19:25.722 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-21T15:19:25.722 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy 2024-09-21T15:19:25.722 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-09-21T15:19:25.743 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T15:19:25.754 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T15:19:25.787 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T15:19:25.787 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-21T15:19:25.790 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-09-21T15:19:25.794 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T15:19:25.805 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T15:19:25.805 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-21T15:19:25.835 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-09-21T15:19:25.861 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T15:19:25.913 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T15:19:25.932 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T15:19:25.932 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T15:19:25.932 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T15:19:25.932 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T15:19:25.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:19:25.937 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-21T15:19:25.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:19:25.938 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T15:19:25.985 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-56-generic 2024-09-21T15:19:25.985 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.15.0-56-generic 2024-09-21T15:19:25.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T15:19:25.986 INFO:teuthology.orchestra.run.smithi107.stdout:5.15.0-56-generic 2024-09-21T15:19:25.987 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.15.0-56-generic 2024-09-21T15:19:25.987 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2024-09-21T15:19:26.117 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T15:19:26.119 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-21T15:19:26.303 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T15:19:26.391 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:19:26.395 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-21T15:19:26.409 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:19:26.448 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T15:19:26.491 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-21T15:19:26.522 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T15:19:26.624 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T15:19:26.628 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T15:19:26.631 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T15:19:26.631 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T15:19:26.631 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T15:19:26.651 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T15:19:26.654 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T15:19:26.655 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T15:19:26.659 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T15:19:26.666 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T15:19:26.667 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T15:19:26.667 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T15:19:26.668 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T15:19:26.668 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T15:19:26.682 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T15:19:26.805 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T15:19:26.844 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-21T15:19:26.850 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-21T15:19:26.942 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T15:19:26.986 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-21T15:19:27.148 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T15:19:27.182 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-21T15:19:27.266 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-21T15:19:27.282 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-21T15:19:27.291 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-21T15:19:27.291 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-21T15:19:27.297 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T15:19:27.311 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T15:19:27.312 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T15:19:27.313 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T15:19:27.342 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-21T15:19:27.356 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-21T15:19:27.357 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-21T15:19:27.359 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-21T15:19:27.386 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T15:19:27.388 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-21T15:19:27.432 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-21T15:19:27.446 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T15:19:27.446 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T15:19:27.450 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-21T15:19:27.450 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-21T15:19:27.450 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-21T15:19:27.458 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-21T15:19:27.458 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-21T15:19:27.470 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T15:19:27.474 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-21T15:19:27.477 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-21T15:19:27.482 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T15:19:27.485 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-21T15:19:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T15:19:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-21T15:19:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T15:19:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-21T15:19:27.489 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T15:19:27.489 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-21T15:19:27.490 INFO:teuthology.orchestra.run.smithi107.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T15:19:27.496 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-21T15:19:27.501 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-21T15:19:27.506 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-21T15:19:27.506 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-21T15:19:27.508 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-21T15:19:27.542 INFO:teuthology.orchestra.run.smithi107.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T15:19:27.542 INFO:teuthology.orchestra.run.smithi107.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T15:19:27.543 INFO:teuthology.orchestra.run.smithi107.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T15:19:27.572 INFO:teuthology.orchestra.run.smithi107.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T15:19:27.572 INFO:teuthology.orchestra.run.smithi107.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T15:19:27.573 INFO:teuthology.orchestra.run.smithi107.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T15:19:27.573 INFO:teuthology.orchestra.run.smithi107.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T15:19:27.574 INFO:teuthology.orchestra.run.smithi107.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T15:19:27.584 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-21T15:19:27.584 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-21T15:19:27.586 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-21T15:19:27.587 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-21T15:19:27.587 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-21T15:19:27.587 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-21T15:19:27.588 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-21T15:19:27.623 INFO:teuthology.orchestra.run.smithi107.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T15:19:27.655 INFO:teuthology.orchestra.run.smithi107.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-21T15:19:27.656 INFO:teuthology.orchestra.run.smithi107.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-21T15:19:27.669 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-21T15:19:27.669 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-21T15:19:32.974 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 16.3 MB in 2s (7,460 kB/s) 2024-09-21T15:19:33.510 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 16.3 MB in 3s (5,865 kB/s) 2024-09-21T15:19:34.489 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:19:34.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T15:19:34.619 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:19:34.831 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T15:19:34.831 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T15:19:35.011 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:19:35.024 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:19:35.024 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:19:35.024 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:19:35.024 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:19:35.024 INFO:teuthology.orchestra.run.smithi170.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T15:19:35.025 INFO:teuthology.orchestra.run.smithi170.stdout: python2.7-minimal 2024-09-21T15:19:35.025 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:19:35.026 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T15:19:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T15:19:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T15:19:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T15:19:35.028 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T15:19:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T15:19:35.028 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T15:19:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2024-09-21T15:19:35.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T15:19:35.080 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T15:19:35.080 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T15:19:35.080 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T15:19:35.080 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T15:19:35.081 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2024-09-21T15:19:35.082 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T15:19:35.090 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:19:35.262 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T15:19:35.262 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 113 MB of archives. 2024-09-21T15:19:35.262 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T15:19:35.262 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T15:19:35.302 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-21T15:19:35.303 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-21T15:19:35.480 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:19:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:19:35.480 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:19:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:19:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-21T15:19:35.481 INFO:teuthology.orchestra.run.smithi107.stdout: python2.7-minimal 2024-09-21T15:19:35.481 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:19:35.482 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2024-09-21T15:19:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T15:19:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-21T15:19:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-21T15:19:35.484 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2024-09-21T15:19:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-21T15:19:35.484 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2024-09-21T15:19:35.484 INFO:teuthology.orchestra.run.smithi107.stdout: thermald 2024-09-21T15:19:35.529 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2024-09-21T15:19:35.529 INFO:teuthology.orchestra.run.smithi107.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-21T15:19:35.530 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-21T15:19:35.530 INFO:teuthology.orchestra.run.smithi107.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-21T15:19:35.531 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages will be upgraded: 2024-09-21T15:19:35.531 INFO:teuthology.orchestra.run.smithi107.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-21T15:19:35.582 INFO:teuthology.orchestra.run.smithi107.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T15:19:35.583 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 113 MB of archives. 2024-09-21T15:19:35.583 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-21T15:19:35.583 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-21T15:19:36.062 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T15:19:36.163 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T15:19:36.660 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-21T15:19:36.966 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T15:19:36.966 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T15:19:36.966 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T15:19:37.062 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-21T15:19:37.094 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T15:19:37.193 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T15:19:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 113 MB in 2s (68.5 MB/s) 2024-09-21T15:19:37.772 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T15:19:39.198 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-21T15:19:39.198 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-21T15:19:39.198 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-21T15:19:39.589 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-21T15:19:39.634 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-21T15:19:39.846 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T15:19:39.850 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:40.035 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 113 MB in 5s (24.9 MB/s) 2024-09-21T15:19:40.136 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:40.215 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-21T15:19:42.292 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-21T15:19:42.295 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T15:19:42.297 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:42.317 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:42.394 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:42.412 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:42.890 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T15:19:42.910 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:42.948 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:44.713 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-21T15:19:44.735 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:44.812 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:45.267 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-21T15:19:45.287 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:45.325 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:49.990 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:19:50.117 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:19:50.432 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:19:50.537 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:19:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T15:19:50.879 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T15:19:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T15:19:52.391 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:19:52.535 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:19:52.867 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:19:52.988 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:19:53.308 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-21T15:19:53.331 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-21T15:19:53.375 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T15:19:56.997 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T15:19:57.034 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:19:57.072 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:19:59.917 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:19:59.970 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-21T15:20:00.008 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-21T15:20:00.039 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:20:00.053 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:00.390 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T15:20:00.511 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:00.629 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T15:20:00.791 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:02.160 INFO:teuthology.orchestra.run.smithi107.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:02.160 INFO:teuthology.orchestra.run.smithi107.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:02.160 INFO:teuthology.orchestra.run.smithi107.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:02.160 INFO:teuthology.orchestra.run.smithi107.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:02.775 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:03.781 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-21T15:20:03.907 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-21T15:20:04.226 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T15:20:04.290 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-21T15:20:04.361 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T15:20:04.428 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:04.487 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:04.529 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-21T15:20:04.658 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:05.914 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:05.980 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T15:20:05.980 INFO:teuthology.orchestra.run.smithi107.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:06.005 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:06.005 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:06.006 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:06.006 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:06.168 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:07.578 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-21T15:20:07.695 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-21T15:20:07.830 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:09.291 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-21T15:20:09.358 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-21T15:20:09.358 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T15:20:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub' 2024-09-21T15:20:20.516 INFO:teuthology.orchestra.run.smithi107.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T15:20:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:Generating grub configuration file ... 2024-09-21T15:20:20.850 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:20.870 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:21.189 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:21.195 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:21.268 INFO:teuthology.orchestra.run.smithi107.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T15:20:21.274 INFO:teuthology.orchestra.run.smithi107.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T15:20:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T15:20:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T15:20:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T15:20:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-09-21T15:20:21.975 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2024-09-21T15:20:22.043 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2024-09-21T15:20:22.043 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2024-09-21T15:20:22.043 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2024-09-21T15:20:22.043 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 21 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.15.0.122.122 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2024-09-21T15:20:22.044 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2024-09-21T15:20:22.044 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T15:20:22.045 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T15:20:22.045 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T15:20:22.045 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T15:20:22.045 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T15:20:22.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get clean 2024-09-21T15:20:22.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-21T15:20:22.327 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:20:22.327 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T15:20:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T15:20:22.355 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T15:20:23.520 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-21T15:20:23.520 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2024-09-21T15:20:23.541 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T15:20:23.563 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2024-09-21T15:20:23.791 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:20:23.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T15:20:23.874 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:20:23.939 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:23.958 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:24.087 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-21T15:20:24.087 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-21T15:20:24.281 INFO:teuthology.orchestra.run.smithi107.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T15:20:24.281 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:20:24.281 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:20:24.281 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:20:24.281 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:20:24.282 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T15:20:24.282 INFO:teuthology.orchestra.run.smithi107.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T15:20:24.282 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:20:24.324 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T15:20:24.326 DEBUG:teuthology.orchestra.run.smithi107:> dpkg -s linux-image-generic 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Package: linux-image-generic 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Status: install ok installed 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Priority: optional 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Section: kernel 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Installed-Size: 21 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: amd64 2024-09-21T15:20:24.338 INFO:teuthology.orchestra.run.smithi107.stdout:Source: linux-meta 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:Version: 5.15.0.122.122 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:Recommends: thermald 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:Description: Generic Linux kernel image 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout: This package will always depend on the latest generic kernel image 2024-09-21T15:20:24.339 INFO:teuthology.orchestra.run.smithi107.stdout: available. 2024-09-21T15:20:24.339 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-21T15:20:24.385 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.0fWQoEBcqG 2024-09-21T15:20:24.385 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub/grub.cfg /tmp/tmp.0fWQoEBcqG 2024-09-21T15:20:24.387 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:24.393 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:24.440 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.0fWQoEBcqG 2024-09-21T15:20:24.464 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T15:20:24.470 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T15:20:24.557 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.0fWQoEBcqG is 10KB 2024-09-21T15:20:24.569 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.0fWQoEBcqG 2024-09-21T15:20:24.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T15:20:24.628 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:20:24.628 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T15:20:24.628 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T15:20:24.658 INFO:teuthology.orchestra.run.smithi170.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T15:20:24.658 INFO:teuthology.orchestra.run.smithi170.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T15:20:24.658 INFO:teuthology.orchestra.run.smithi170.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T15:20:24.691 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T15:20:24.692 DEBUG:teuthology.orchestra.run.smithi107:> sudo update-grub 2024-09-21T15:20:24.697 INFO:teuthology.orchestra.run.smithi170.stdout:done 2024-09-21T15:20:25.213 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T15:20:25.240 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T15:20:25.241 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T15:20:25.242 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-21T15:20:25.242 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T15:20:25.242 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T15:20:25.242 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T15:20:25.242 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T15:20:25.242 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-09-21T15:20:25.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T15:20:25.678 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T15:20:25.679 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:20:25.749 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T15:20:25.831 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T15:20:26.099 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub' 2024-09-21T15:20:26.100 INFO:teuthology.orchestra.run.smithi107.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T15:20:26.103 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-21T15:20:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:26.302 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:26.593 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:26.598 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T15:20:26.643 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T15:20:26.829 INFO:teuthology.orchestra.run.smithi107.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T15:20:26.829 INFO:teuthology.orchestra.run.smithi107.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T15:20:26.829 INFO:teuthology.orchestra.run.smithi107.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T15:20:26.847 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-21T15:20:26.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-21T15:20:27.112 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:20:27.131 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-21T15:20:27.194 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:20:27.407 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T15:20:27.407 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T15:20:27.581 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T15:20:27.582 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T15:20:27.583 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:20:27.620 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-21T15:20:27.622 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-09-21T15:20:27.637 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.122.122 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-09-21T15:20:27.638 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-09-21T15:20:27.638 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-21T15:20:27.684 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.rcOyaptvs8 2024-09-21T15:20:27.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rcOyaptvs8 2024-09-21T15:20:27.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.rcOyaptvs8 2024-09-21T15:20:27.851 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.rcOyaptvs8 is 10KB 2024-09-21T15:20:27.863 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.rcOyaptvs8 2024-09-21T15:20:27.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-21T15:20:27.919 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:20:27.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-21T15:20:27.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-21T15:20:27.988 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-21T15:20:27.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2024-09-21T15:20:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2024-09-21T15:20:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-21T15:20:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-09-21T15:20:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-21T15:20:29.634 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-21T15:20:29.961 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-21T15:20:29.966 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-21T15:20:30.010 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-21T15:20:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-21T15:20:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-21T15:20:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-21T15:20:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-21T15:20:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-09-21T15:20:30.164 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-09-21T15:20:56.878 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T15:20:56.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:20:56.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:21:00.213 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T15:21:00.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:21:00.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:21:31.375 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T15:21:40.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:21:40.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:21:58.767 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T15:22:00.217 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T15:22:09.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:22:09.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:22:09.228 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-09-21T15:22:10.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:22:10.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:22:12.819 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-21T15:22:21.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:22:21.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:22:21.501 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T15:22:22.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:22:22.077 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-21T15:22:22.077 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-09-21T15:22:22.125 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-122-generic 2024-09-21T15:22:22.125 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T15:22:22.125 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T15:22:22.125 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T15:22:23.126 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T15:22:23.126 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T15:22:23.211 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-09-21T15:22:23.243 DEBUG:teuthology.parallel:result is None 2024-09-21T15:22:27.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:22:27.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:22:28.202 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-21T15:22:28.788 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:22:28.788 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-21T15:22:28.788 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-21T15:22:28.836 INFO:teuthology.orchestra.run.smithi107.stdout:5.15.0-122-generic 2024-09-21T15:22:28.836 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-21T15:22:28.836 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T15:22:28.836 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-21T15:22:29.836 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T15:22:29.837 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T15:22:30.194 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-21T15:22:30.213 DEBUG:teuthology.parallel:result is None 2024-09-21T15:22:30.213 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T15:22:30.220 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T15:22:30.220 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T15:22:30.222 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T15:22:30.228 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T15:22:30.243 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T15:22:30.253 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T15:22:30.253 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T15:22:30.273 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T15:22:30.296 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T15:22:30.302 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T15:22:30.302 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T15:22:30.333 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T15:22:30.351 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T15:22:30.352 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T15:22:30.359 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T15:22:30.361 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T15:22:30.364 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T15:22:30.370 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T15:22:30.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T15:22:30.405 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T15:22:30.418 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T15:22:30.425 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T15:22:30.426 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T15:22:30.461 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T15:22:30.466 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T15:22:30.512 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T15:22:30.560 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:22:30.560 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T15:22:30.630 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T15:22:30.636 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T15:22:30.685 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:22:30.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T15:22:30.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-21T15:22:30.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-09-21T15:22:30.830 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T15:22:30.837 INFO:teuthology.task.internal:Starting timer... 2024-09-21T15:22:30.837 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T15:22:30.863 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T15:22:30.870 DEBUG:teuthology.task.selinux:Excluding smithi107: OS 'ubuntu' does not support SELinux 2024-09-21T15:22:30.871 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2024-09-21T15:22:30.871 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T15:22:30.871 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-21T15:22:30.871 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T15:22:30.871 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T15:22:30.879 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T15:22:30.881 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T15:22:30.976 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T15:22:30.985 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T15:22:30.985 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T15:29:57.555 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-09-21T15:29:57.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:29:57.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:29:57.637 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-21T15:29:57.715 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T15:29:57.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:29:57.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T15:29:57.794 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-09-21T15:29:57.875 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-09-21T15:29:57.875 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T15:29:57.885 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T15:29:57.885 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T15:29:57.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:29:57.887 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T15:29:57.888 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:29:57.914 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T15:29:57.914 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Command line: ntpd -gq 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: ---------------------------------------------------- 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: corporation. Support and training for ntp-4 are 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: available at https://www.nwtime.org/support 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: ---------------------------------------------------- 2024-09-21T15:29:57.915 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: proto: precision = 0.044 usec (-24) 2024-09-21T15:29:57.916 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: basedate set to 2022-02-04 2024-09-21T15:29:57.916 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: gps base set to 2022-02-06 (week 2196) 2024-09-21T15:29:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T15:29:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen normally on 3 enp3s0f1 172.21.15.107:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen normally on 4 lo [::1]:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:47eb%5]:123 2024-09-21T15:29:57.917 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:29:57 ntpd[18492]: Listening on routing socket on fd #22 for interface updates 2024-09-21T15:29:57.946 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-21T15:29:57.946 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Command line: ntpd -gq 2024-09-21T15:29:57.946 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: ---------------------------------------------------- 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: ntp-4 is maintained by Network Time Foundation, 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: corporation. Support and training for ntp-4 are 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: available at https://www.nwtime.org/support 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: ---------------------------------------------------- 2024-09-21T15:29:57.947 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: proto: precision = 0.062 usec (-24) 2024-09-21T15:29:57.948 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: basedate set to 2022-02-04 2024-09-21T15:29:57.948 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: gps base set to 2022-02-06 (week 2196) 2024-09-21T15:29:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T15:29:57.948 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-21T15:29:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-21T15:29:57.949 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: restrict ::: KOD does nothing without LIMITED. 2024-09-21T15:29:57.949 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen and drop on 0 v6wildcard [::]:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen normally on 2 lo 127.0.0.1:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen normally on 4 lo [::1]:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2024-09-21T15:29:57.950 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:29:57 ntpd[18457]: Listening on routing socket on fd #22 for interface updates 2024-09-21T15:29:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:29:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:29:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:29:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:29:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:29:58.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:29:58 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:29:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:29:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:29:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:29:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:29:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:29:59.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:29:59 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:00.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:00 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:00.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:00.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:00.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:00.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:00.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:00 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:01.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:01.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:01 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:02 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:02.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:02 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:03.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:03.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:03.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:03.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:03.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:03.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:03 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:04 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.948 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stdout:21 Sep 15:30:04 ntpd[18457]: ntpd: time slew -0.001156 s 2024-09-21T15:30:04.949 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew -0.001156s 2024-09-21T15:30:04.950 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T15:30:04.950 INFO:teuthology.orchestra.run.smithi170.stderr:21 Sep 15:30:04 ntpd[18457]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:05.005 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:05.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:05.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:05.915 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-21T15:30:05.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240921: Permission denied 2024-09-21T15:30:05.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-21T15:30:05.916 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240921: Permission denied 2024-09-21T15:30:05.916 INFO:teuthology.orchestra.run.smithi107.stdout:21 Sep 15:30:05 ntpd[18492]: ntpd: time slew -0.001846 s 2024-09-21T15:30:05.916 INFO:teuthology.orchestra.run.smithi107.stdout:ntpd: time slew -0.001846s 2024-09-21T15:30:05.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-21T15:30:05.917 INFO:teuthology.orchestra.run.smithi107.stderr:21 Sep 15:30:05 ntpd[18492]: can't open /var/log/ntpstats/loopstats.20240921: Permission denied 2024-09-21T15:30:06.072 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:30:06.072 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2024-09-21T15:30:06.072 INFO:teuthology.orchestra.run.smithi107.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:06.072 INFO:teuthology.orchestra.run.smithi107.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:06.073 INFO:teuthology.orchestra.run.smithi107.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:06.073 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:30:06.074 INFO:teuthology.run_tasks:Running task install... 2024-09-21T15:30:06.085 DEBUG:teuthology.task.install:project ceph 2024-09-21T15:30:06.086 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-21T15:30:06.086 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-21T15:30:06.086 INFO:teuthology.task.install:Using flavor: default 2024-09-21T15:30:06.091 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T15:30:06.091 INFO:teuthology.task.install:extra packages: [] 2024-09-21T15:30:06.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-key list | grep Ceph 2024-09-21T15:30:06.092 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2024-09-21T15:30:06.150 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T15:30:06.175 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T15:30:06.175 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T15:30:06.176 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T15:30:06.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T15:30:06.180 INFO:teuthology.orchestra.run.smithi107.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-21T15:30:06.210 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-21T15:30:06.210 INFO:teuthology.orchestra.run.smithi107.stdout:uid [ unknown] Ceph.com (release key) 2024-09-21T15:30:06.210 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-21T15:30:06.210 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T15:30:06.329 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default/ 2024-09-21T15:30:06.329 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:30:06.346 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default/ 2024-09-21T15:30:06.347 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:30:06.440 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:30:06.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T15:30:06.455 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:30:06.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-21T15:30:06.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-09-21T15:30:06.469 DEBUG:teuthology.orchestra.run.smithi107:> sudo apt-get update 2024-09-21T15:30:06.642 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T15:30:06.678 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T15:30:06.690 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-21T15:30:06.700 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T15:30:06.711 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-21T15:30:06.723 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T15:30:06.733 INFO:teuthology.orchestra.run.smithi107.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-21T15:30:06.753 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:30:06.753 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-21T15:30:06.754 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-21T15:30:06.760 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-21T15:30:06.764 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-21T15:30:06.781 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,437 B] 2024-09-21T15:30:06.781 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,437 B] 2024-09-21T15:30:06.825 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T15:30:06.839 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-21T15:30:06.908 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T15:30:06.921 INFO:teuthology.orchestra.run.smithi107.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-21T15:30:07.062 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 28.8 kB in 0s (69.7 kB/s) 2024-09-21T15:30:07.098 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 28.8 kB in 0s (61.7 kB/s) 2024-09-21T15:30:08.056 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:30:08.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1jammy cephadm=18.2.4-855-g19e2a2be-1jammy ceph-mds=18.2.4-855-g19e2a2be-1jammy ceph-mgr=18.2.4-855-g19e2a2be-1jammy ceph-common=18.2.4-855-g19e2a2be-1jammy ceph-fuse=18.2.4-855-g19e2a2be-1jammy ceph-test=18.2.4-855-g19e2a2be-1jammy ceph-volume=18.2.4-855-g19e2a2be-1jammy radosgw=18.2.4-855-g19e2a2be-1jammy python3-rados=18.2.4-855-g19e2a2be-1jammy python3-rgw=18.2.4-855-g19e2a2be-1jammy python3-cephfs=18.2.4-855-g19e2a2be-1jammy python3-rbd=18.2.4-855-g19e2a2be-1jammy libcephfs2=18.2.4-855-g19e2a2be-1jammy libcephfs-dev=18.2.4-855-g19e2a2be-1jammy librados2=18.2.4-855-g19e2a2be-1jammy librbd1=18.2.4-855-g19e2a2be-1jammy rbd-fuse=18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:30:08.091 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:30:08.108 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1jammy cephadm=18.2.4-855-g19e2a2be-1jammy ceph-mds=18.2.4-855-g19e2a2be-1jammy ceph-mgr=18.2.4-855-g19e2a2be-1jammy ceph-common=18.2.4-855-g19e2a2be-1jammy ceph-fuse=18.2.4-855-g19e2a2be-1jammy ceph-test=18.2.4-855-g19e2a2be-1jammy ceph-volume=18.2.4-855-g19e2a2be-1jammy radosgw=18.2.4-855-g19e2a2be-1jammy python3-rados=18.2.4-855-g19e2a2be-1jammy python3-rgw=18.2.4-855-g19e2a2be-1jammy python3-cephfs=18.2.4-855-g19e2a2be-1jammy python3-rbd=18.2.4-855-g19e2a2be-1jammy libcephfs2=18.2.4-855-g19e2a2be-1jammy libcephfs-dev=18.2.4-855-g19e2a2be-1jammy librados2=18.2.4-855-g19e2a2be-1jammy librbd1=18.2.4-855-g19e2a2be-1jammy rbd-fuse=18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:30:08.136 INFO:teuthology.orchestra.run.smithi107.stdout:Reading package lists... 2024-09-21T15:30:08.162 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-09-21T15:30:08.347 INFO:teuthology.orchestra.run.smithi107.stdout:Building dependency tree... 2024-09-21T15:30:08.348 INFO:teuthology.orchestra.run.smithi107.stdout:Reading state information... 2024-09-21T15:30:08.349 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-09-21T15:30:08.350 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-09-21T15:30:08.535 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:30:08.535 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:30:08.535 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:30:08.535 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:30:08.536 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T15:30:08.536 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T15:30:08.536 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:30:08.536 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-09-21T15:30:08.537 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T15:30:08.537 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T15:30:08.537 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-21T15:30:08.537 INFO:teuthology.orchestra.run.smithi170.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T15:30:08.538 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T15:30:08.539 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: python3-watchdog 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-09-21T15:30:08.540 INFO:teuthology.orchestra.run.smithi170.stdout: btrfs-tools 2024-09-21T15:30:08.546 INFO:teuthology.orchestra.run.smithi107.stdout:The following packages were automatically installed and are no longer required: 2024-09-21T15:30:08.546 INFO:teuthology.orchestra.run.smithi107.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-21T15:30:08.546 INFO:teuthology.orchestra.run.smithi107.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-21T15:30:08.546 INFO:teuthology.orchestra.run.smithi107.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-21T15:30:08.547 INFO:teuthology.orchestra.run.smithi107.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-21T15:30:08.547 INFO:teuthology.orchestra.run.smithi107.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-21T15:30:08.547 INFO:teuthology.orchestra.run.smithi107.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-21T15:30:08.548 INFO:teuthology.orchestra.run.smithi107.stdout:The following additional packages will be installed: 2024-09-21T15:30:08.548 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-21T15:30:08.548 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-21T15:30:08.548 INFO:teuthology.orchestra.run.smithi107.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-21T15:30:08.549 INFO:teuthology.orchestra.run.smithi107.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-21T15:30:08.549 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-21T15:30:08.549 INFO:teuthology.orchestra.run.smithi107.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-21T15:30:08.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-21T15:30:08.549 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T15:30:08.550 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout:Suggested packages: 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout: python3-watchdog 2024-09-21T15:30:08.551 INFO:teuthology.orchestra.run.smithi107.stdout:Recommended packages: 2024-09-21T15:30:08.552 INFO:teuthology.orchestra.run.smithi107.stdout: btrfs-tools 2024-09-21T15:30:08.639 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-09-21T15:30:08.639 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T15:30:08.639 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T15:30:08.639 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T15:30:08.640 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-21T15:30:08.640 INFO:teuthology.orchestra.run.smithi170.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T15:30:08.640 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-21T15:30:08.640 INFO:teuthology.orchestra.run.smithi170.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T15:30:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T15:30:08.642 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T15:30:08.642 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T15:30:08.642 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T15:30:08.642 INFO:teuthology.orchestra.run.smithi170.stdout: socat xmlstarlet 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout:The following NEW packages will be installed: 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-21T15:30:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-21T15:30:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-21T15:30:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-21T15:30:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-21T15:30:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-21T15:30:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-21T15:30:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: socat xmlstarlet 2024-09-21T15:30:08.785 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 109 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T15:30:08.785 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 163 MB of archives. 2024-09-21T15:30:08.786 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-21T15:30:08.786 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1jammy [3,452 kB] 2024-09-21T15:30:08.803 INFO:teuthology.orchestra.run.smithi107.stdout:0 upgraded, 109 newly installed, 0 to remove and 332 not upgraded. 2024-09-21T15:30:08.803 INFO:teuthology.orchestra.run.smithi107.stdout:Need to get 163 MB of archives. 2024-09-21T15:30:08.803 INFO:teuthology.orchestra.run.smithi107.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-21T15:30:08.803 INFO:teuthology.orchestra.run.smithi107.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1jammy [3,452 kB] 2024-09-21T15:30:08.831 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-21T15:30:08.843 INFO:teuthology.orchestra.run.smithi107.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-21T15:30:08.985 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1jammy [3,108 kB] 2024-09-21T15:30:09.005 INFO:teuthology.orchestra.run.smithi107.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1jammy [3,108 kB] 2024-09-21T15:30:09.062 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1jammy [683 kB] 2024-09-21T15:30:09.087 INFO:teuthology.orchestra.run.smithi107.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1jammy [683 kB] 2024-09-21T15:30:09.097 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1jammy [357 kB] 2024-09-21T15:30:09.103 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1jammy [32.8 kB] 2024-09-21T15:30:09.104 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1jammy [180 kB] 2024-09-21T15:30:09.107 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1jammy [63.8 kB] 2024-09-21T15:30:09.109 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1jammy [334 kB] 2024-09-21T15:30:09.110 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T15:30:09.114 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1jammy [5,934 kB] 2024-09-21T15:30:09.125 INFO:teuthology.orchestra.run.smithi107.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-21T15:30:09.129 INFO:teuthology.orchestra.run.smithi107.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1jammy [357 kB] 2024-09-21T15:30:09.137 INFO:teuthology.orchestra.run.smithi107.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1jammy [32.8 kB] 2024-09-21T15:30:09.138 INFO:teuthology.orchestra.run.smithi107.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1jammy [180 kB] 2024-09-21T15:30:09.141 INFO:teuthology.orchestra.run.smithi107.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1jammy [63.8 kB] 2024-09-21T15:30:09.143 INFO:teuthology.orchestra.run.smithi107.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1jammy [334 kB] 2024-09-21T15:30:09.150 INFO:teuthology.orchestra.run.smithi107.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1jammy [5,934 kB] 2024-09-21T15:30:09.283 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T15:30:09.297 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T15:30:09.297 INFO:teuthology.orchestra.run.smithi107.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-21T15:30:09.311 INFO:teuthology.orchestra.run.smithi107.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-21T15:30:09.315 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1jammy [112 kB] 2024-09-21T15:30:09.318 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1jammy [445 kB] 2024-09-21T15:30:09.326 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1jammy [22.0 MB] 2024-09-21T15:30:09.347 INFO:teuthology.orchestra.run.smithi107.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1jammy [112 kB] 2024-09-21T15:30:09.350 INFO:teuthology.orchestra.run.smithi107.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1jammy [445 kB] 2024-09-21T15:30:09.358 INFO:teuthology.orchestra.run.smithi107.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1jammy [22.0 MB] 2024-09-21T15:30:09.370 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T15:30:09.383 INFO:teuthology.orchestra.run.smithi107.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-21T15:30:09.564 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T15:30:09.578 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T15:30:09.580 INFO:teuthology.orchestra.run.smithi107.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-21T15:30:09.589 INFO:teuthology.orchestra.run.smithi107.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-21T15:30:09.605 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T15:30:09.613 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T15:30:09.614 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T15:30:09.615 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T15:30:09.617 INFO:teuthology.orchestra.run.smithi107.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-21T15:30:09.625 INFO:teuthology.orchestra.run.smithi107.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-21T15:30:09.626 INFO:teuthology.orchestra.run.smithi107.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-21T15:30:09.627 INFO:teuthology.orchestra.run.smithi107.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-21T15:30:09.632 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T15:30:09.633 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T15:30:09.634 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T15:30:09.644 INFO:teuthology.orchestra.run.smithi107.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-21T15:30:09.646 INFO:teuthology.orchestra.run.smithi107.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-21T15:30:09.646 INFO:teuthology.orchestra.run.smithi107.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-21T15:30:09.647 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T15:30:09.663 INFO:teuthology.orchestra.run.smithi107.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-21T15:30:09.719 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T15:30:09.719 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T15:30:09.719 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T15:30:09.720 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T15:30:09.721 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T15:30:09.722 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T15:30:09.729 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T15:30:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-21T15:30:09.731 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T15:30:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-21T15:30:09.731 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T15:30:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-21T15:30:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-21T15:30:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-21T15:30:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-21T15:30:09.739 INFO:teuthology.orchestra.run.smithi107.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-21T15:30:09.745 INFO:teuthology.orchestra.run.smithi107.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-21T15:30:09.746 INFO:teuthology.orchestra.run.smithi107.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-21T15:30:09.810 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T15:30:09.812 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T15:30:09.812 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T15:30:09.813 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T15:30:09.815 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T15:30:09.817 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T15:30:09.826 INFO:teuthology.orchestra.run.smithi107.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-21T15:30:09.828 INFO:teuthology.orchestra.run.smithi107.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-21T15:30:09.828 INFO:teuthology.orchestra.run.smithi107.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-21T15:30:09.829 INFO:teuthology.orchestra.run.smithi107.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-21T15:30:09.831 INFO:teuthology.orchestra.run.smithi107.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-21T15:30:09.833 INFO:teuthology.orchestra.run.smithi107.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-21T15:30:09.834 INFO:teuthology.orchestra.run.smithi107.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T15:30:09.891 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-21T15:30:09.902 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T15:30:09.903 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T15:30:09.904 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T15:30:09.907 INFO:teuthology.orchestra.run.smithi107.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-21T15:30:09.908 INFO:teuthology.orchestra.run.smithi107.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-21T15:30:09.909 INFO:teuthology.orchestra.run.smithi107.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-21T15:30:09.973 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T15:30:09.975 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T15:30:09.979 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T15:30:09.981 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T15:30:09.984 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T15:30:09.985 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T15:30:09.989 INFO:teuthology.orchestra.run.smithi107.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-21T15:30:09.992 INFO:teuthology.orchestra.run.smithi107.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-21T15:30:09.996 INFO:teuthology.orchestra.run.smithi107.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-21T15:30:09.997 INFO:teuthology.orchestra.run.smithi107.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-21T15:30:10.000 INFO:teuthology.orchestra.run.smithi107.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-21T15:30:10.001 INFO:teuthology.orchestra.run.smithi107.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-21T15:30:10.006 INFO:teuthology.orchestra.run.smithi107.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T15:30:10.054 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-21T15:30:10.055 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T15:30:10.057 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T15:30:10.062 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T15:30:10.071 INFO:teuthology.orchestra.run.smithi107.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-21T15:30:10.121 INFO:teuthology.orchestra.run.smithi107.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-21T15:30:10.123 INFO:teuthology.orchestra.run.smithi107.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1jammy [5,006 kB] 2024-09-21T15:30:10.126 INFO:teuthology.orchestra.run.smithi107.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-21T15:30:10.169 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1jammy [5,006 kB] 2024-09-21T15:30:10.216 INFO:teuthology.orchestra.run.smithi107.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1jammy [241 kB] 2024-09-21T15:30:10.218 INFO:teuthology.orchestra.run.smithi107.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1jammy [130 kB] 2024-09-21T15:30:10.224 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T15:30:10.226 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T15:30:10.228 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T15:30:10.228 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T15:30:10.230 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T15:30:10.230 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T15:30:10.231 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T15:30:10.234 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T15:30:10.235 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T15:30:10.239 INFO:teuthology.orchestra.run.smithi107.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1jammy [1,494 kB] 2024-09-21T15:30:10.252 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1jammy [241 kB] 2024-09-21T15:30:10.264 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1jammy [130 kB] 2024-09-21T15:30:10.266 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1jammy [1,494 kB] 2024-09-21T15:30:10.267 INFO:teuthology.orchestra.run.smithi107.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-21T15:30:10.269 INFO:teuthology.orchestra.run.smithi107.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-21T15:30:10.271 INFO:teuthology.orchestra.run.smithi107.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-21T15:30:10.272 INFO:teuthology.orchestra.run.smithi107.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-21T15:30:10.274 INFO:teuthology.orchestra.run.smithi107.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-21T15:30:10.274 INFO:teuthology.orchestra.run.smithi107.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-21T15:30:10.275 INFO:teuthology.orchestra.run.smithi107.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-21T15:30:10.279 INFO:teuthology.orchestra.run.smithi107.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-21T15:30:10.279 INFO:teuthology.orchestra.run.smithi107.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-21T15:30:10.285 INFO:teuthology.orchestra.run.smithi107.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1jammy [6,085 kB] 2024-09-21T15:30:10.309 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T15:30:10.315 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1jammy [6,085 kB] 2024-09-21T15:30:10.349 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T15:30:10.352 INFO:teuthology.orchestra.run.smithi107.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-21T15:30:10.353 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T15:30:10.353 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T15:30:10.390 INFO:teuthology.orchestra.run.smithi107.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-21T15:30:10.394 INFO:teuthology.orchestra.run.smithi107.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-21T15:30:10.394 INFO:teuthology.orchestra.run.smithi107.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-21T15:30:10.410 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T15:30:10.410 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T15:30:10.411 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T15:30:10.412 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T15:30:10.413 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T15:30:10.413 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T15:30:10.453 INFO:teuthology.orchestra.run.smithi107.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-21T15:30:10.454 INFO:teuthology.orchestra.run.smithi107.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-21T15:30:10.455 INFO:teuthology.orchestra.run.smithi107.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-21T15:30:10.456 INFO:teuthology.orchestra.run.smithi107.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-21T15:30:10.456 INFO:teuthology.orchestra.run.smithi107.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-21T15:30:10.457 INFO:teuthology.orchestra.run.smithi107.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-21T15:30:10.472 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T15:30:10.473 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T15:30:10.475 INFO:teuthology.orchestra.run.smithi107.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1jammy [22.4 MB] 2024-09-21T15:30:10.479 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T15:30:10.515 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1jammy [22.4 MB] 2024-09-21T15:30:10.516 INFO:teuthology.orchestra.run.smithi107.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-21T15:30:10.517 INFO:teuthology.orchestra.run.smithi107.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-21T15:30:10.523 INFO:teuthology.orchestra.run.smithi107.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-21T15:30:10.554 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T15:30:10.557 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T15:30:10.559 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T15:30:10.568 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T15:30:10.574 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T15:30:10.584 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T15:30:10.590 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T15:30:10.597 INFO:teuthology.orchestra.run.smithi107.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-21T15:30:10.600 INFO:teuthology.orchestra.run.smithi107.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-21T15:30:10.602 INFO:teuthology.orchestra.run.smithi107.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-21T15:30:10.610 INFO:teuthology.orchestra.run.smithi107.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-21T15:30:10.615 INFO:teuthology.orchestra.run.smithi107.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-21T15:30:10.625 INFO:teuthology.orchestra.run.smithi107.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-21T15:30:10.632 INFO:teuthology.orchestra.run.smithi107.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-21T15:30:10.635 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T15:30:10.639 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T15:30:10.655 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T15:30:10.679 INFO:teuthology.orchestra.run.smithi107.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-21T15:30:10.682 INFO:teuthology.orchestra.run.smithi107.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-21T15:30:10.698 INFO:teuthology.orchestra.run.smithi107.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-21T15:30:10.717 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T15:30:10.718 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T15:30:10.760 INFO:teuthology.orchestra.run.smithi107.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-21T15:30:10.762 INFO:teuthology.orchestra.run.smithi107.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-21T15:30:11.310 INFO:teuthology.orchestra.run.smithi107.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1jammy [14.2 kB] 2024-09-21T15:30:11.311 INFO:teuthology.orchestra.run.smithi107.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1jammy [835 kB] 2024-09-21T15:30:11.327 INFO:teuthology.orchestra.run.smithi107.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1jammy [2,159 kB] 2024-09-21T15:30:11.358 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1jammy [14.2 kB] 2024-09-21T15:30:11.359 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1jammy [835 kB] 2024-09-21T15:30:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1jammy [240 kB] 2024-09-21T15:30:11.368 INFO:teuthology.orchestra.run.smithi107.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1jammy [145 kB] 2024-09-21T15:30:11.371 INFO:teuthology.orchestra.run.smithi107.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1jammy [2,233 kB] 2024-09-21T15:30:11.375 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1jammy [2,159 kB] 2024-09-21T15:30:11.394 INFO:teuthology.orchestra.run.smithi107.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1jammy [8,625 kB] 2024-09-21T15:30:11.413 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1jammy [240 kB] 2024-09-21T15:30:11.417 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1jammy [145 kB] 2024-09-21T15:30:11.419 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1jammy [2,233 kB] 2024-09-21T15:30:11.452 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1jammy [8,625 kB] 2024-09-21T15:30:11.649 INFO:teuthology.orchestra.run.smithi107.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1jammy [14.3 kB] 2024-09-21T15:30:11.649 INFO:teuthology.orchestra.run.smithi107.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1jammy [42.8 MB] 2024-09-21T15:30:11.770 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1jammy [14.3 kB] 2024-09-21T15:30:11.770 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1jammy [42.8 MB] 2024-09-21T15:30:13.205 INFO:teuthology.orchestra.run.smithi107.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1jammy [122 kB] 2024-09-21T15:30:13.208 INFO:teuthology.orchestra.run.smithi107.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1jammy [38.3 kB] 2024-09-21T15:30:13.209 INFO:teuthology.orchestra.run.smithi107.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1jammy [11.7 MB] 2024-09-21T15:30:13.342 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1jammy [122 kB] 2024-09-21T15:30:13.345 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1jammy [38.3 kB] 2024-09-21T15:30:13.346 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1jammy [11.7 MB] 2024-09-21T15:30:13.366 INFO:teuthology.orchestra.run.smithi107.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1jammy [93.1 kB] 2024-09-21T15:30:13.504 INFO:teuthology.orchestra.run.smithi170.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1jammy [93.1 kB] 2024-09-21T15:30:13.918 INFO:teuthology.orchestra.run.smithi107.stdout:Fetched 163 MB in 5s (34.6 MB/s) 2024-09-21T15:30:14.035 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-21T15:30:14.074 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 163 MB in 5s (33.5 MB/s) 2024-09-21T15:30:14.120 INFO:teuthology.orchestra.run.smithi107.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T15:30:14.125 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-21T15:30:14.214 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-21T15:30:14.217 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-21T15:30:14.301 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-21T15:30:14.306 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-21T15:30:14.388 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-21T15:30:14.520 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T15:30:14.546 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T15:30:14.578 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T15:30:14.666 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-21T15:30:14.692 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-21T15:30:14.732 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T15:30:14.926 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T15:30:14.950 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T15:30:14.990 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T15:30:15.027 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-21T15:30:15.053 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-21T15:30:15.094 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T15:30:15.360 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T15:30:15.386 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T15:30:15.422 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-21T15:30:15.427 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T15:30:15.448 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-21T15:30:15.488 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T15:30:15.789 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T15:30:15.815 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:15.864 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:15.900 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-21T15:30:15.926 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:15.975 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:16.276 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T15:30:16.302 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:16.342 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:16.442 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-21T15:30:16.468 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:16.512 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:16.679 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T15:30:16.705 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:16.745 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:16.865 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-21T15:30:16.891 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-21T15:30:16.931 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:30:17.057 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T15:30:17.072 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T15:30:17.107 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T15:30:17.259 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-21T15:30:17.286 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-21T15:30:17.325 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T15:30:17.427 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librados2. 2024-09-21T15:30:17.453 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:17.493 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:17.620 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librados2. 2024-09-21T15:30:17.647 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:17.687 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:17.946 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librbd1. 2024-09-21T15:30:17.973 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../009-librbd1_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:18.021 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:18.131 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librbd1. 2024-09-21T15:30:18.158 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-librbd1_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:18.198 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:18.416 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T15:30:18.443 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:18.474 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:18.651 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2024-09-21T15:30:18.677 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:18.717 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:18.751 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rados. 2024-09-21T15:30:18.767 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../011-python3-rados_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:18.810 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:18.970 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2024-09-21T15:30:18.985 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-python3-rados_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:19.020 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:19.130 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T15:30:19.156 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:19.196 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:19.306 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-21T15:30:19.332 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:19.372 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:19.457 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T15:30:19.472 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:19.516 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:19.625 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2024-09-21T15:30:19.640 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:19.683 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:19.835 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T15:30:19.862 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:19.902 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:20.003 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-21T15:30:20.030 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:20.069 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:20.222 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T15:30:20.248 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T15:30:20.280 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T15:30:20.372 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-21T15:30:20.398 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-21T15:30:20.439 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T15:30:20.575 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T15:30:20.601 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T15:30:20.641 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T15:30:20.700 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2024-09-21T15:30:20.726 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-21T15:30:20.767 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-21T15:30:20.902 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T15:30:20.929 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:20.969 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:21.036 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2024-09-21T15:30:21.063 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:21.102 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:21.306 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T15:30:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T15:30:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T15:30:21.447 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-21T15:30:21.473 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-21T15:30:21.514 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T15:30:21.700 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package librgw2. 2024-09-21T15:30:21.726 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:21.766 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:21.825 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2024-09-21T15:30:21.851 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:21.883 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:22.244 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T15:30:22.259 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:22.294 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:22.352 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2024-09-21T15:30:22.379 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:22.419 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:22.647 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T15:30:22.673 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T15:30:22.714 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T15:30:22.772 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-21T15:30:22.799 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-21T15:30:22.838 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T15:30:23.025 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T15:30:23.051 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:23.092 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:23.141 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2024-09-21T15:30:23.168 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:23.208 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:23.487 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-common. 2024-09-21T15:30:23.513 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:23.545 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:23.561 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2024-09-21T15:30:23.588 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:23.627 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:24.842 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2024-09-21T15:30:24.868 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:24.876 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-base. 2024-09-21T15:30:24.903 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:24.917 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:24.952 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:25.463 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T15:30:25.489 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T15:30:25.529 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T15:30:25.531 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-21T15:30:25.557 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-21T15:30:25.597 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-21T15:30:25.884 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T15:30:25.910 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T15:30:25.950 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T15:30:26.082 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cheroot. 2024-09-21T15:30:26.108 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-21T15:30:26.149 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T15:30:26.460 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T15:30:26.486 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T15:30:26.526 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T15:30:26.570 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-21T15:30:26.596 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-21T15:30:26.636 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-21T15:30:26.754 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T15:30:26.769 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T15:30:26.804 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T15:30:26.915 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-21T15:30:26.941 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-21T15:30:26.981 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-21T15:30:27.099 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T15:30:27.125 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T15:30:27.165 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T15:30:27.267 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-21T15:30:27.293 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-21T15:30:27.334 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-21T15:30:27.468 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T15:30:27.494 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T15:30:27.535 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T15:30:27.662 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-tempora. 2024-09-21T15:30:27.688 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-21T15:30:27.728 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-21T15:30:27.804 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-portend. 2024-09-21T15:30:27.819 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-21T15:30:27.854 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T15:30:27.990 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-portend. 2024-09-21T15:30:28.017 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-21T15:30:28.057 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-21T15:30:28.124 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T15:30:28.150 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T15:30:28.190 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T15:30:28.360 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-21T15:30:28.386 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-21T15:30:28.426 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-21T15:30:28.518 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T15:30:28.545 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T15:30:28.585 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T15:30:28.729 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-21T15:30:28.755 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-21T15:30:28.796 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-21T15:30:28.947 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T15:30:28.974 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T15:30:29.014 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T15:30:29.141 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-natsort. 2024-09-21T15:30:29.168 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-21T15:30:29.208 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-21T15:30:29.283 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T15:30:29.309 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T15:30:29.341 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T15:30:29.528 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-logutils. 2024-09-21T15:30:29.554 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-21T15:30:29.594 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-21T15:30:29.610 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T15:30:29.637 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T15:30:29.677 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T15:30:29.964 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-21T15:30:29.982 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2024-09-21T15:30:29.990 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-21T15:30:30.008 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T15:30:30.039 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-21T15:30:30.049 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T15:30:30.367 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-mako. 2024-09-21T15:30:30.394 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T15:30:30.394 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-21T15:30:30.420 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T15:30:30.433 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T15:30:30.460 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T15:30:30.730 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T15:30:30.756 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T15:30:30.770 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-21T15:30:30.796 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T15:30:30.797 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-21T15:30:30.837 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-21T15:30:31.116 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2024-09-21T15:30:31.142 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T15:30:31.182 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-21T15:30:31.182 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T15:30:31.208 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-21T15:30:31.257 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-21T15:30:31.511 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T15:30:31.537 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T15:30:31.581 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T15:30:31.660 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webob. 2024-09-21T15:30:31.686 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-21T15:30:31.735 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T15:30:31.889 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T15:30:31.915 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T15:30:31.955 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T15:30:32.104 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-waitress. 2024-09-21T15:30:32.131 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-21T15:30:32.182 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T15:30:32.233 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2024-09-21T15:30:32.260 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T15:30:32.300 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T15:30:32.491 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-tempita. 2024-09-21T15:30:32.517 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-21T15:30:32.558 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T15:30:32.620 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T15:30:32.646 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T15:30:32.686 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T15:30:32.894 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-paste. 2024-09-21T15:30:32.920 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-21T15:30:32.964 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T15:30:32.969 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T15:30:32.991 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T15:30:33.031 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T15:30:33.275 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T15:30:33.289 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-21T15:30:33.291 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T15:30:33.304 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-21T15:30:33.325 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T15:30:33.339 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T15:30:33.592 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-21T15:30:33.618 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-21T15:30:33.628 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T15:30:33.655 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T15:30:33.658 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-21T15:30:33.695 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T15:30:33.961 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-webtest. 2024-09-21T15:30:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-21T15:30:34.028 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-21T15:30:34.050 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T15:30:34.076 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T15:30:34.108 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T15:30:34.331 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pecan. 2024-09-21T15:30:34.357 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-21T15:30:34.398 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T15:30:34.428 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-idna. 2024-09-21T15:30:34.454 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T15:30:34.494 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T15:30:34.761 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-certifi. 2024-09-21T15:30:34.787 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-21T15:30:34.789 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T15:30:34.815 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T15:30:34.827 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-21T15:30:34.856 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T15:30:35.149 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-idna. 2024-09-21T15:30:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-21T15:30:35.176 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests. 2024-09-21T15:30:35.202 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T15:30:35.224 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T15:30:35.242 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T15:30:35.537 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T15:30:35.543 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-urllib3. 2024-09-21T15:30:35.563 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T15:30:35.570 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-21T15:30:35.603 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T15:30:35.610 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T15:30:35.906 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T15:30:35.922 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:35.965 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:35.971 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-requests. 2024-09-21T15:30:35.998 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-21T15:30:36.038 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T15:30:36.349 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-21T15:30:36.375 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-21T15:30:36.416 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T15:30:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T15:30:36.521 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:36.561 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:36.710 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-21T15:30:36.741 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:36.881 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T15:30:36.907 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T15:30:36.935 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:36.947 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T15:30:37.217 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T15:30:37.243 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T15:30:37.275 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T15:30:37.331 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-21T15:30:37.357 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:37.398 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:37.578 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T15:30:37.605 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:37.645 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:37.717 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-21T15:30:37.743 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-21T15:30:37.784 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-21T15:30:38.065 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T15:30:38.086 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-openssl. 2024-09-21T15:30:38.091 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:38.113 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-21T15:30:38.131 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:38.153 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-21T15:30:38.439 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr. 2024-09-21T15:30:38.465 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:38.497 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:38.693 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T15:30:38.720 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:38.760 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:38.900 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mon. 2024-09-21T15:30:38.927 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:38.967 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:39.595 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-osd. 2024-09-21T15:30:39.622 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:39.661 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:39.772 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2024-09-21T15:30:39.798 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-ceph_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:39.847 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:40.184 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T15:30:40.210 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:40.257 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:40.603 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T15:30:40.629 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:40.669 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:40.706 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph. 2024-09-21T15:30:40.722 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../060-ceph_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:40.765 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:41.067 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-fuse. 2024-09-21T15:30:41.093 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:41.133 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:41.512 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mds. 2024-09-21T15:30:41.538 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:41.578 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:41.748 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2024-09-21T15:30:41.775 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-cephadm_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:41.814 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:42.126 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T15:30:42.152 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T15:30:42.192 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T15:30:42.511 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T15:30:42.527 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T15:30:42.570 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T15:30:42.857 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package cephadm. 2024-09-21T15:30:42.884 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../063-cephadm_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:42.915 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:43.201 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-21T15:30:43.228 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-21T15:30:43.268 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-21T15:30:43.550 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-babel. 2024-09-21T15:30:43.578 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T15:30:43.579 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-21T15:30:43.594 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T15:30:43.608 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T15:30:43.629 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T15:30:43.903 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T15:30:43.930 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T15:30:43.961 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T15:30:44.282 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T15:30:44.308 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:44.348 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:44.652 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T15:30:44.678 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T15:30:44.692 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-babel. 2024-09-21T15:30:44.718 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T15:30:44.719 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-21T15:30:44.758 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T15:30:45.030 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2024-09-21T15:30:45.056 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-21T15:30:45.095 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-jinja2. 2024-09-21T15:30:45.096 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T15:30:45.122 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-21T15:30:45.161 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T15:30:45.426 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T15:30:45.452 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T15:30:45.492 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T15:30:45.506 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-21T15:30:45.532 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:45.572 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:45.794 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-toml. 2024-09-21T15:30:45.821 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-21T15:30:45.861 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T15:30:45.918 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pluggy. 2024-09-21T15:30:45.944 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-21T15:30:45.984 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-21T15:30:46.181 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T15:30:46.207 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T15:30:46.247 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T15:30:46.295 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-py. 2024-09-21T15:30:46.322 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-21T15:30:46.362 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-21T15:30:46.690 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-21T15:30:46.717 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-21T15:30:46.757 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-21T15:30:46.784 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T15:30:46.812 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T15:30:46.867 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T15:30:47.104 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2024-09-21T15:30:47.119 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T15:30:47.154 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T15:30:47.235 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-toml. 2024-09-21T15:30:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-21T15:30:47.301 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-21T15:30:47.457 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T15:30:47.473 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:47.532 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:47.587 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pytest. 2024-09-21T15:30:47.614 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-21T15:30:47.653 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T15:30:47.974 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-21T15:30:48.000 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-21T15:30:48.057 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-21T15:30:48.335 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-routes. 2024-09-21T15:30:48.361 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-21T15:30:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T15:30:48.721 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-21T15:30:48.748 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:48.754 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T15:30:48.770 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T15:30:48.787 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:48.812 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T15:30:49.258 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T15:30:49.273 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T15:30:49.308 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T15:30:49.661 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T15:30:49.688 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T15:30:49.728 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T15:30:50.031 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T15:30:50.058 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T15:30:50.076 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-21T15:30:50.097 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T15:30:50.103 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-21T15:30:50.142 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T15:30:50.621 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-joblib. 2024-09-21T15:30:50.647 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-21T15:30:50.679 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T15:30:50.765 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T15:30:50.792 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:50.831 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-21T15:30:51.076 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-21T15:30:51.116 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-21T15:30:51.435 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-sklearn. 2024-09-21T15:30:51.462 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-21T15:30:51.501 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T15:30:51.809 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T15:30:51.836 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T15:30:51.876 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T15:30:52.251 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-21T15:30:52.263 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T15:30:52.267 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:52.290 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T15:30:52.301 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:52.330 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T15:30:52.592 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T15:30:52.607 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T15:30:52.642 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T15:30:52.928 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T15:30:52.955 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-21T15:30:52.995 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T15:30:53.229 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-cachetools. 2024-09-21T15:30:53.256 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-21T15:30:53.295 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-21T15:30:53.307 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T15:30:53.334 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T15:30:53.374 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T15:30:53.683 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-21T15:30:53.693 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T15:30:53.709 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-21T15:30:53.720 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T15:30:53.749 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-21T15:30:53.760 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T15:30:54.047 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T15:30:54.074 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T15:30:54.094 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-21T15:30:54.113 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T15:30:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-21T15:30:54.161 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T15:30:54.383 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T15:30:54.399 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T15:30:54.441 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T15:30:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-rsa. 2024-09-21T15:30:54.482 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-21T15:30:54.521 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-21T15:30:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-google-auth. 2024-09-21T15:30:54.819 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-21T15:30:54.850 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-21T15:30:55.055 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T15:30:55.072 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:55.105 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:55.128 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-21T15:30:55.155 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-21T15:30:55.195 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T15:30:55.408 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T15:30:55.436 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T15:30:55.475 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T15:30:55.506 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-websocket. 2024-09-21T15:30:55.533 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-21T15:30:55.572 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-21T15:30:55.820 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T15:30:55.847 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T15:30:55.886 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T15:30:55.901 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-21T15:30:55.928 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-21T15:30:55.967 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T15:30:56.147 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2024-09-21T15:30:56.175 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T15:30:56.213 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T15:30:56.416 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2024-09-21T15:30:56.433 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T15:30:56.466 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T15:30:56.597 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-21T15:30:56.611 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:56.647 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:56.745 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T15:30:56.757 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T15:30:56.794 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T15:30:56.975 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-21T15:30:57.003 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-21T15:30:57.041 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T15:30:57.149 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2024-09-21T15:30:57.177 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-ceph-test_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:57.215 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:57.336 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-21T15:30:57.352 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T15:30:57.386 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T15:30:57.647 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package jq. 2024-09-21T15:30:57.674 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-21T15:30:57.713 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-21T15:30:57.974 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package socat. 2024-09-21T15:30:57.990 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-21T15:30:58.032 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-21T15:30:58.361 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package xmlstarlet. 2024-09-21T15:30:58.388 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-21T15:30:58.427 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-21T15:30:58.773 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-test. 2024-09-21T15:30:58.800 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../096-ceph-test_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:58.839 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:59.061 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T15:30:59.077 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:30:59.111 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:59.390 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T15:30:59.406 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:30:59.440 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:30:59.794 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T15:30:59.821 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T15:30:59.860 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T15:31:00.223 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T15:31:00.251 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T15:31:00.290 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T15:31:00.710 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package ceph-volume. 2024-09-21T15:31:00.718 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T15:31:00.737 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-855-g19e2a2be-1jammy_all.deb ... 2024-09-21T15:31:00.745 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T15:31:00.767 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:00.784 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T15:31:01.055 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-21T15:31:01.082 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:31:01.105 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T15:31:01.121 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:01.132 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T15:31:01.171 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T15:31:01.466 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package nvme-cli. 2024-09-21T15:31:01.483 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T15:31:01.493 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-21T15:31:01.499 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T15:31:01.532 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T15:31:01.541 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T15:31:01.862 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-21T15:31:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-21T15:31:01.912 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-21T15:31:02.228 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T15:31:02.256 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T15:31:02.295 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T15:31:02.340 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pastescript. 2024-09-21T15:31:02.367 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-21T15:31:02.398 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-21T15:31:02.748 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T15:31:02.775 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T15:31:02.805 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T15:31:02.943 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-psutil. 2024-09-21T15:31:02.970 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-21T15:31:03.009 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-21T15:31:03.033 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T15:31:03.049 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T15:31:03.092 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T15:31:03.362 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pygments. 2024-09-21T15:31:03.390 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-21T15:31:03.429 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T15:31:03.796 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2024-09-21T15:31:03.823 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-radosgw_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:31:03.854 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:03.899 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-21T15:31:03.927 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-21T15:31:03.965 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-21T15:31:04.243 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package python3-simplejson. 2024-09-21T15:31:04.259 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-21T15:31:04.293 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-21T15:31:04.587 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-21T15:31:04.590 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T15:31:04.603 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-21T15:31:04.618 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:31:04.645 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-21T15:31:04.657 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:04.952 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T15:31:05.203 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T15:31:05.304 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T15:31:05.433 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package radosgw. 2024-09-21T15:31:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../107-radosgw_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:31:05.499 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:05.565 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T15:31:05.920 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T15:31:06.210 INFO:teuthology.orchestra.run.smithi107.stdout:Selecting previously unselected package rbd-fuse. 2024-09-21T15:31:06.226 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-855-g19e2a2be-1jammy_amd64.deb ... 2024-09-21T15:31:06.260 INFO:teuthology.orchestra.run.smithi107.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:06.315 INFO:teuthology.orchestra.run.smithi170.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T15:31:06.415 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:06.563 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-21T15:31:06.814 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-21T15:31:06.932 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-21T15:31:07.192 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-21T15:31:07.224 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2024-09-21T15:31:07.539 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-21T15:31:07.597 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T15:31:07.890 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T15:31:07.967 INFO:teuthology.orchestra.run.smithi107.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-21T15:31:08.133 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T15:31:08.259 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T15:31:08.260 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:08.527 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T15:31:08.778 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T15:31:08.817 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user cephadm....done 2024-09-21T15:31:08.904 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T15:31:08.957 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-21T15:31:09.216 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T15:31:09.241 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-21T15:31:09.501 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-21T15:31:09.576 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T15:31:09.602 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-21T15:31:09.844 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-21T15:31:09.861 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T15:31:10.087 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-21T15:31:10.104 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T15:31:10.213 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-21T15:31:10.355 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T15:31:10.532 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-21T15:31:10.606 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T15:31:10.900 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-21T15:31:10.982 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:11.168 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-21T15:31:11.242 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T15:31:11.368 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T15:31:11.419 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-21T15:31:11.619 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T15:31:11.670 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-21T15:31:11.728 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T15:31:11.921 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-21T15:31:12.079 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T15:31:12.172 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:12.206 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T15:31:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-21T15:31:12.448 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T15:31:12.573 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-21T15:31:12.716 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T15:31:12.833 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-21T15:31:12.959 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-21T15:31:13.034 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T15:31:13.310 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-21T15:31:13.411 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-21T15:31:13.670 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-21T15:31:13.736 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T15:31:13.938 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-21T15:31:13.996 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T15:31:14.122 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T15:31:14.231 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-21T15:31:14.248 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T15:31:14.549 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T15:31:14.874 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-21T15:31:14.934 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T15:31:15.134 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-21T15:31:15.185 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T15:31:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-21T15:31:15.344 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-21T15:31:15.561 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T15:31:15.637 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-21T15:31:15.846 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T15:31:15.996 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-21T15:31:16.181 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T15:31:16.239 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-21T15:31:16.424 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T15:31:16.558 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:16.607 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-21T15:31:16.851 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T15:31:16.925 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-21T15:31:17.169 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T15:31:17.268 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-21T15:31:17.511 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-21T15:31:17.611 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:17.887 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-21T15:31:18.359 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:18.364 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-21T15:31:18.493 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T15:31:18.752 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T15:31:18.879 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T15:31:19.005 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T15:31:19.297 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T15:31:19.548 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T15:31:19.744 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:19.792 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:19.878 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-21T15:31:19.918 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-21T15:31:20.036 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T15:31:20.129 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-21T15:31:20.255 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-21T15:31:20.337 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T15:31:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-21T15:31:20.588 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T15:31:20.683 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-21T15:31:20.839 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T15:31:20.958 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-21T15:31:21.082 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T15:31:21.209 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:21.319 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-21T15:31:21.358 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T15:31:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-21T15:31:21.668 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T15:31:21.754 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-21T15:31:21.785 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T15:31:22.014 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-21T15:31:22.053 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T15:31:22.179 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T15:31:22.273 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-21T15:31:22.464 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T15:31:22.541 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-21T15:31:22.792 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-21T15:31:22.865 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T15:31:23.076 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-21T15:31:23.141 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T15:31:23.177 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-21T15:31:23.453 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-21T15:31:23.516 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T15:31:23.554 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-21T15:31:23.753 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T15:31:23.849 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-21T15:31:23.862 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T15:31:24.150 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-21T15:31:24.255 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T15:31:24.401 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-21T15:31:24.515 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T15:31:24.766 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:24.802 INFO:teuthology.orchestra.run.smithi107.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-21T15:31:24.892 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T15:31:25.053 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-21T15:31:25.180 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-21T15:31:25.218 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T15:31:25.511 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T15:31:25.564 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-21T15:31:25.771 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T15:31:25.832 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-21T15:31:25.897 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T15:31:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-21T15:31:26.164 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T15:31:26.217 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-21T15:31:26.416 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T15:31:26.518 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-21T15:31:26.684 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T15:31:26.811 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-21T15:31:27.079 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-21T15:31:27.093 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T15:31:27.205 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-21T15:31:27.387 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T15:31:27.490 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-21T15:31:27.722 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T15:31:27.757 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-21T15:31:27.831 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:27.941 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:28.075 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-21T15:31:28.076 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:28.201 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:28.327 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:28.452 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-21T15:31:28.530 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T15:31:28.744 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-21T15:31:28.799 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:28.933 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.060 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.096 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-21T15:31:29.223 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.272 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T15:31:29.272 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T15:31:29.349 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.475 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.576 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.685 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.702 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.812 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:29.905 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-21T15:31:29.937 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T15:31:30.190 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:30.316 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:30.425 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:30.623 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T15:31:30.623 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-21T15:31:31.135 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.176 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.261 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.301 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.387 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-21T15:31:31.428 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.554 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:31.680 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:32.190 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2024-09-21T15:31:32.575 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2024-09-21T15:31:32.608 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:32.709 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:32.744 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2024-09-21T15:31:32.761 INFO:teuthology.orchestra.run.smithi170.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T15:31:32.827 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:32.897 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T15:31:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:33.095 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:33.208 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T15:31:33.646 INFO:teuthology.orchestra.run.smithi107.stdout:Adding group ceph....done 2024-09-21T15:31:33.673 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:33.775 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:34.007 INFO:teuthology.orchestra.run.smithi107.stdout:Adding system user ceph....done 2024-09-21T15:31:34.158 INFO:teuthology.orchestra.run.smithi107.stdout:Setting system user ceph properties....done 2024-09-21T15:31:34.172 INFO:teuthology.orchestra.run.smithi107.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-21T15:31:34.244 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T15:31:34.244 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T15:31:34.312 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-21T15:31:34.662 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-21T15:31:34.701 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:35.018 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T15:31:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:35.298 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:35.487 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:35.719 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T15:31:35.719 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T15:31:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T15:31:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-21T15:31:36.163 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:36.233 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:36.374 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T15:31:36.374 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T15:31:36.546 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-21T15:31:36.823 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:37.051 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:37.079 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T15:31:37.079 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T15:31:37.343 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T15:31:37.343 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-21T15:31:37.550 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:37.659 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:37.810 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:37.836 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:38.023 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T15:31:38.023 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T15:31:38.052 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T15:31:38.053 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-21T15:31:38.495 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:38.554 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:38.646 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:38.789 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:38.866 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T15:31:38.866 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-21T15:31:38.915 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:39.252 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T15:31:39.356 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:39.482 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:39.633 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:39.844 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T15:31:39.844 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-21T15:31:40.326 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:40.468 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:40.594 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:40.633 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T15:31:40.745 INFO:teuthology.orchestra.run.smithi107.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1jammy) ... 2024-09-21T15:31:41.091 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-21T15:31:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T15:31:41.226 DEBUG:teuthology.parallel:result is None 2024-09-21T15:31:42.216 INFO:teuthology.orchestra.run.smithi107.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-21T15:31:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-21T15:31:42.680 DEBUG:teuthology.parallel:result is None 2024-09-21T15:31:42.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T15:31:42.829 DEBUG:teuthology.orchestra.run.smithi107:> dpkg-query -W -f '${Version}' ceph 2024-09-21T15:31:42.844 INFO:teuthology.orchestra.run.smithi107.stdout:18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:31:42.844 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:31:42.845 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1jammy is installed. 2024-09-21T15:31:42.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-21T15:31:42.984 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2024-09-21T15:31:43.003 INFO:teuthology.orchestra.run.smithi170.stdout:18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:31:43.003 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1jammy 2024-09-21T15:31:43.003 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1jammy is installed. 2024-09-21T15:31:43.005 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T15:31:43.005 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:43.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T15:31:43.019 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:43.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T15:31:43.057 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T15:31:43.058 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:43.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T15:31:43.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T15:31:43.128 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:43.128 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T15:31:43.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T15:31:43.195 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T15:31:43.196 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:43.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T15:31:43.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T15:31:43.264 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:43.264 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T15:31:43.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T15:31:43.332 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T15:31:43.333 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:43.333 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T15:31:43.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T15:31:43.399 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:43.399 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T15:31:43.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T15:31:43.468 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T15:31:43.546 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T15:31:43.547 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2024-09-21T15:31:43.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2024-09-21T15:31:43.562 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T15:31:43.562 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T15:31:43.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T15:31:43.618 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T15:31:43.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T15:31:43.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T15:31:43.682 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T15:31:43.682 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T15:31:43.683 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909985', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4623, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'sleep_before_teardown': 0, 'subset': '80/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7', 'targets': {'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVx/pr2e3QZWtwdmhJBT4lNtEZOqZ+0MFN3M773+9qypXkCpkcrNMfStw7xxB2MMmljwS/opx1YSiqLPlRtIKc=', 'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHocvVSLooJ9vSsVNgY50udOabyuZbBbmcdmJUJkjjUxzJcqH57RE6CfjG13QxU98Emu4yJbNwQEDcivIp8AqL0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_22:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T15:31:43.683 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T15:31:43.710 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T15:31:43.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T15:31:43.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T15:31:43.777 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:43.777 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T15:31:43.817 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T15:31:43.817 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-21T15:31:43.865 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T15:31:43.865 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:43.865 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-21T15:31:43.865 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:43.865 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 15:26:27.668246974 +0000 2024-09-21T15:31:43.866 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 15:26:27.440249012 +0000 2024-09-21T15:31:43.866 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 15:26:27.440249012 +0000 2024-09-21T15:31:43.866 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-21T15:31:43.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T15:31:43.919 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T15:31:43.919 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T15:31:43.919 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000383565 s, 1.3 MB/s 2024-09-21T15:31:43.921 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T15:31:43.970 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 15:26:27.928244652 +0000 2024-09-21T15:31:44.017 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 15:26:27.924244688 +0000 2024-09-21T15:31:44.018 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 15:26:27.924244688 +0000 2024-09-21T15:31:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-21T15:31:44.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T15:31:44.072 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T15:31:44.072 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T15:31:44.072 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000417934 s, 1.2 MB/s 2024-09-21T15:31:44.074 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T15:31:44.122 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 15:26:28.428240186 +0000 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 15:26:28.424240221 +0000 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 15:26:28.424240221 +0000 2024-09-21T15:31:44.169 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-21T15:31:44.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T15:31:44.223 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T15:31:44.223 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T15:31:44.223 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00040964 s, 1.2 MB/s 2024-09-21T15:31:44.225 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T15:31:44.274 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 15:26:29.164233612 +0000 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 15:26:28.920235791 +0000 2024-09-21T15:31:44.321 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 15:26:28.920235791 +0000 2024-09-21T15:31:44.322 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-09-21T15:31:44.322 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T15:31:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T15:31:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T15:31:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000393351 s, 1.3 MB/s 2024-09-21T15:31:44.377 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T15:31:44.426 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:44.426 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:44.427 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T15:31:44.432 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T15:31:44.433 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 15:26:25.678579081 +0000 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 15:26:25.446581154 +0000 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 15:26:25.446581154 +0000 2024-09-21T15:31:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T15:31:44.482 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T15:31:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T15:31:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T15:31:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000331727 s, 1.5 MB/s 2024-09-21T15:31:44.536 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T15:31:44.588 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 15:26:26.190574508 +0000 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 15:26:25.922576902 +0000 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 15:26:25.922576902 +0000 2024-09-21T15:31:44.637 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T15:31:44.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T15:31:44.690 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T15:31:44.690 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T15:31:44.691 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000251005 s, 2.0 MB/s 2024-09-21T15:31:44.692 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T15:31:44.742 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 15:26:26.646570435 +0000 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 15:26:26.414572506 +0000 2024-09-21T15:31:44.790 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 15:26:26.414572506 +0000 2024-09-21T15:31:44.791 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T15:31:44.791 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T15:31:44.843 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T15:31:44.843 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T15:31:44.843 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000273488 s, 1.9 MB/s 2024-09-21T15:31:44.844 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T15:31:44.894 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-09-21T15:31:44.940 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-09-21 15:26:26.874568399 +0000 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-09-21 15:26:26.870568433 +0000 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-09-21 15:26:26.870568433 +0000 2024-09-21T15:31:44.941 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-09-21T15:31:44.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T15:31:44.995 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-09-21T15:31:44.995 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-09-21T15:31:44.995 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000243619 s, 2.1 MB/s 2024-09-21T15:31:44.996 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T15:31:45.045 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:45.046 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T15:31:45.046 INFO:tasks.ceph:Generating config... 2024-09-21T15:31:45.047 INFO:tasks.ceph:[client] rbd cache = False 2024-09-21T15:31:45.047 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T15:31:45.047 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T15:31:45.047 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T15:31:45.047 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T15:31:45.047 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T15:31:45.047 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T15:31:45.047 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T15:31:45.047 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T15:31:45.047 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T15:31:45.048 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-21T15:31:45.049 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T15:31:45.049 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T15:31:45.049 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T15:31:45.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T15:31:45.083 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T15:31:45.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T15:31:45.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T15:31:45.233 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.107'), ('mon.c', '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]'), ('mon.b', '172.21.15.170')] 2024-09-21T15:31:45.233 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790],172.21.15.170', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T15:31:45.235 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:45.235 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T15:31:45.281 DEBUG:teuthology.orchestra.run.smithi107:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.107 --addv c '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]' --add b 172.21.15.170 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool: generated fsid 09582c7a-ec34-4064-810b-b65e59e21b91 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = pacific 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 0 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:fsid 09582c7a-ec34-4064-810b-b65e59e21b91 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:last_changed 2024-09-21T15:31:45.367070+0000 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-09-21T15:31:45.367070+0000 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:min_mon_release 16 (pacific) 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:election_strategy: 1 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-09-21T15:31:45.367 INFO:teuthology.orchestra.run.smithi107.stdout:1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-09-21T15:31:45.368 INFO:teuthology.orchestra.run.smithi107.stdout:2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-09-21T15:31:45.368 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T15:31:45.369 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T15:31:45.417 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 09582c7a-ec34-4064-810b-b65e59e21b91... 2024-09-21T15:31:45.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T15:31:45.462 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T15:31:45.491 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-21T15:31:45.491 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2024-09-21T15:31:45.491 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T15:31:45.491 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2024-09-21T15:31:45.491 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.492 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2024-09-21T15:31:45.492 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.492 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2024-09-21T15:31:45.492 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.492 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T15:31:45.493 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2024-09-21T15:31:45.494 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.494 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T15:31:45.494 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T15:31:45.494 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.494 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T15:31:45.495 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790],172.21.15.170" 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon client directed command retry = 5 2024-09-21T15:31:45.496 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on pool no app = False 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject socket failures = 5000 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 09582c7a-ec34-4064-810b-b65e59e21b91 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2024-09-21T15:31:45.497 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2024-09-21T15:31:45.498 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: bdev async discard = True 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: bdev enable discard = True 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 1/20 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 1/20 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 4/10 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T15:31:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd blocked scrub grace period = 3600 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd max markdown count = 1000 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd memory target = 1610612736 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: osd shutdown pgref assert = True 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-21T15:31:45.500 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.501 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2024-09-21T15:31:45.502 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout: rbd cache = False 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2024-09-21T15:31:45.503 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2024-09-21T15:31:45.505 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.506 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2024-09-21T15:31:45.507 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2024-09-21T15:31:45.508 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790],172.21.15.170" 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: mon client directed command retry = 5 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on pool no app = False 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: ms inject socket failures = 5000 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 09582c7a-ec34-4064-810b-b65e59e21b91 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2024-09-21T15:31:45.509 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.510 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2024-09-21T15:31:45.511 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd blocked scrub grace period = 3600 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd max markdown count = 1000 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd memory target = 1610612736 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: osd shutdown pgref assert = True 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T15:31:45.512 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T15:31:45.513 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2024-09-21T15:31:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout: rbd cache = False 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2024-09-21T15:31:45.515 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2024-09-21T15:31:45.515 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T15:31:45.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T15:31:45.595 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T15:31:45.596 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:45.596 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T15:31:45.603 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:45.603 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T15:31:45.648 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:31:45.648 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:45.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T15:31:45.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T15:31:45.713 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:31:45.713 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:45.760 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:31:45.760 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:45.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T15:31:45.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T15:31:45.784 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:31:45.785 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:45.833 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T15:31:45.833 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T15:31:45.833 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T15:31:45.876 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T15:31:45.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T15:31:45.922 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T15:31:45.934 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T15:31:45.934 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T15:31:45.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T15:31:45.968 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T15:31:45.978 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T15:31:46.013 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T15:31:46.024 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T15:31:46.024 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T15:31:46.024 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T15:31:46.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:46.035 INFO:tasks.ceph:role: osd.0 2024-09-21T15:31:46.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:31:46.036 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:31:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:31:46.096 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:31:46.096 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:31:46.564 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-21T15:31:46.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-21T15:31:46.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T15:31:46.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T15:31:46.712 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:31:46.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:31:46.713 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T15:31:46.769 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:46.769 INFO:tasks.ceph:role: osd.1 2024-09-21T15:31:46.770 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:31:46.770 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T15:31:46.835 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:31:46.836 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:31:46.837 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:31:46.837 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:31:47.267 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-21T15:31:47.269 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-21T15:31:47.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T15:31:47.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T15:31:47.385 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:31:47.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:31:47.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T15:31:47.439 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:47.439 INFO:tasks.ceph:role: osd.2 2024-09-21T15:31:47.439 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:31:47.439 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T15:31:47.500 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:31:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:31:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:31:47.964 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-21T15:31:47.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-21T15:31:47.966 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T15:31:48.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T15:31:48.071 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:31:48.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:31:48.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T15:31:48.128 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T15:31:48.129 INFO:tasks.ceph:role: osd.3 2024-09-21T15:31:48.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:31:48.129 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T15:31:48.192 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:31:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:31:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:31:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:31:48.192 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:31:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:31:48.193 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:31:48.193 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:31:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:31:48.193 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:31:48.648 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-09-21T15:31:48.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-09-21T15:31:48.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T15:31:48.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T15:31:48.823 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:31:48.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:31:48.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:48.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.928+0000 7ff53914b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T15:31:48.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.932+0000 7ff53914b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T15:31:48.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.932+0000 7ff53914b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T15:31:48.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.932+0000 7ff53914b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T15:31:48.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.932+0000 7ff53914b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-21T15:31:48.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:48.932+0000 7ff53914b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T15:31:51.905 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T15:31:51.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T15:31:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T15:31:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T15:31:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T15:31:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-21T15:31:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:52.000+0000 7ffb3fe5a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T15:31:54.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T15:31:55.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:55.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T15:31:55.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T15:31:55.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T15:31:55.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T15:31:55.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-21T15:31:55.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:55.116+0000 7f0568d3c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T15:31:58.084 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T15:31:58.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:31:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.220+0000 7fcedb0377c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T15:31:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.220+0000 7fcedb0377c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T15:31:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.220+0000 7fcedb0377c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T15:31:58.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.220+0000 7fcedb0377c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T15:31:58.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.220+0000 7fcedb0377c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-21T15:31:58.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:31:58.224+0000 7fcedb0377c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T15:32:01.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T15:32:01.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T15:32:01.272 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T15:32:01.272 INFO:tasks.ceph:role: osd.4 2024-09-21T15:32:01.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:32:01.272 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:32:01.329 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:32:01.353 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T15:32:01.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T15:32:01.355 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T15:32:01.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T15:32:01.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:32:01.465 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:32:01.465 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T15:32:01.518 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T15:32:01.518 INFO:tasks.ceph:role: osd.5 2024-09-21T15:32:01.518 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:32:01.518 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T15:32:01.570 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:32:01.571 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:32:01.594 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T15:32:01.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T15:32:01.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T15:32:01.660 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T15:32:01.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:32:01.713 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:32:01.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T15:32:01.768 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T15:32:01.768 INFO:tasks.ceph:role: osd.6 2024-09-21T15:32:01.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:32:01.769 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T15:32:01.823 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:32:01.823 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:32:01.824 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:32:01.848 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T15:32:01.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T15:32:01.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T15:32:01.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T15:32:01.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:32:01.961 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:32:01.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T15:32:02.015 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T15:32:02.015 INFO:tasks.ceph:role: osd.7 2024-09-21T15:32:02.016 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:32:02.016 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T15:32:02.073 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T15:32:02.099 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-09-21T15:32:02.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-09-21T15:32:02.101 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T15:32:02.164 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T15:32:02.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:32:02.218 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-09-21T15:32:02.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-21T15:32:02.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:02.324+0000 7f0b0fbb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T15:32:05.300 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T15:32:05.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-21T15:32:05.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:05.392+0000 7f8a4f2677c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T15:32:08.366 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T15:32:08.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T15:32:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T15:32:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T15:32:08.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T15:32:08.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-21T15:32:08.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:08.504+0000 7f29d603d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T15:32:11.483 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T15:32:11.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T15:32:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T15:32:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T15:32:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T15:32:11.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-21T15:32:11.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:11.620+0000 7fc4e39cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T15:32:14.593 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T15:32:14.649 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T15:32:14.649 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:14.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T15:32:14.664 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:14.664 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T15:32:14.721 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:14.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T15:32:14.775 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:14.775 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T15:32:14.827 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:14.827 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T15:32:14.880 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:14.880 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T15:32:14.893 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:14.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T15:32:14.947 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:14.947 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T15:32:14.999 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:15.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T15:32:15.055 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:15.055 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T15:32:15.107 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:15.107 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T15:32:15.113 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:15.113 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T15:32:15.156 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T15:32:15.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T15:32:15.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T15:32:15.202 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2024-09-21T15:32:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDh5u5mX/w2NBAAp/FUvkQ643V4yy4H8qw70w== 2024-09-21T15:32:15.202 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2024-09-21T15:32:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDk5u5mltZpNxAALcY8dWLQzAQhaucOhpRQcw== 2024-09-21T15:32:15.202 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDo5u5mVTwwABAAD5ernsbzaVf4VnaP4Ve/xg== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDr5u5mf5MKBxAAWGLb5zEpG1ycIGAURSTPIQ== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDu5u5memJbDRAAtaD01eGkpFry199KHxS6nw== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDh5u5m31IQNxAAX+06oWSgHAiqB4EWKM1XeQ== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDy5u5m3gF6ExAALx6jhAArp2dAkbb4Hqaklw== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD15u5mwuSeFxAAtTC8qhSreyixa/XOLw7PwQ== 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2024-09-21T15:32:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD45u5mOAM8HhAAAoq3Bh5hrczp0SV6SmBLtQ== 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQD75u5mU+IwJRAATKdDKMJ2uXpygqKnNWfWBg== 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDh5u5m11W2ORAAqqYu2nsRvMlSen6CFK6bpg== 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2024-09-21T15:32:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDi5u5m0bHeABAAKdfN+mKMDcYmt1mRSegiqA== 2024-09-21T15:32:15.210 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh5u5mX/w2NBAAp/FUvkQ643V4yy4H8qw70w== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDk5u5mltZpNxAALcY8dWLQzAQhaucOhpRQcw== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDo5u5mVTwwABAAD5ernsbzaVf4VnaP4Ve/xg== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDr5u5mf5MKBxAAWGLb5zEpG1ycIGAURSTPIQ== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDu5u5memJbDRAAtaD01eGkpFry199KHxS6nw== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh5u5m31IQNxAAX+06oWSgHAiqB4EWKM1XeQ== 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2024-09-21T15:32:15.211 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDy5u5m3gF6ExAALx6jhAArp2dAkbb4Hqaklw== 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD15u5mwuSeFxAAtTC8qhSreyixa/XOLw7PwQ== 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD45u5mOAM8HhAAAoq3Bh5hrczp0SV6SmBLtQ== 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD75u5mU+IwJRAATKdDKMJ2uXpygqKnNWfWBg== 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh5u5m11W2ORAAqqYu2nsRvMlSen6CFK6bpg== 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2024-09-21T15:32:15.212 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi5u5m0bHeABAAKdfN+mKMDcYmt1mRSegiqA== 2024-09-21T15:32:15.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T15:32:15.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T15:32:15.301 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.304 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.465 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.545 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.598 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T15:32:15.630 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T15:32:15.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.714 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.798 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.882 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.928 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:15.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T15:32:16.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T15:32:16.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T15:32:16.105 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T15:32:16.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T15:32:16.151 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T15:32:16.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T15:32:16.204 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T15:32:16.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T15:32:16.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T15:32:16.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T15:32:16.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T15:32:16.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T15:32:16.475 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T15:32:16.566 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T15:32:16.577 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:16.579 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T15:32:16.623 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T15:32:16.624 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T15:32:16.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T15:32:16.626 INFO:tasks.ceph.mon.a:Started 2024-09-21T15:32:16.626 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T15:32:16.627 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T15:32:16.629 INFO:tasks.ceph.mon.c:Started 2024-09-21T15:32:16.629 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T15:32:16.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T15:32:16.672 INFO:tasks.ceph.mon.b:Started 2024-09-21T15:32:16.672 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T15:32:16.672 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T15:32:16.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T15:32:16.674 INFO:tasks.ceph.mgr.y:Started 2024-09-21T15:32:16.675 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T15:32:16.675 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T15:32:16.677 INFO:tasks.ceph.mgr.x:Started 2024-09-21T15:32:16.677 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T15:32:16.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T15:32:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-09-21T15:32:17.149 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T15:32:17.149 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T15:32:17.149 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:17.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T15:32:17.161 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:17.161 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T15:32:17.214 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:17.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T15:32:17.234 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:17.232+0000 7f8e16a12180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T15:32:17.244 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T15:32:17.244 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T15:32:17.299 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:17.300 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T15:32:17.302 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:17.300+0000 7fa99bf09180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T15:32:17.313 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:17.313 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T15:32:17.367 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:17.368 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T15:32:17.422 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-09-21T15:32:17.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T15:32:17.477 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new d2f9d147-bbb0-470e-8441-d891316be3ee 0 2024-09-21T15:32:17.759 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:17.756+0000 7f8e16a12180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T15:32:17.810 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:17.808+0000 7fa99bf09180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T15:32:17.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-09-21T15:32:17.848+0000 7f8d1ffff640 0 --2- 172.21.15.170:0/825807141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c004d10 0x7f8d0c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T15:32:17.950 INFO:teuthology.orchestra.run.smithi170.stdout:0 2024-09-21T15:32:17.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new e13f399d-5ddd-47e1-879e-eda2f7ec930f 1 2024-09-21T15:32:17.996 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:17.996+0000 7fa99bf09180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T15:32:18.005 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.004+0000 7f8e16a12180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T15:32:18.167 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-09-21T15:32:18.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 84d52d47-67ab-43ca-a983-5568d9cf1975 2 2024-09-21T15:32:18.427 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-09-21T15:32:18.443 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new adbb6235-d8c3-4b20-a7e4-0046d22ed5fc 3 2024-09-21T15:32:18.556 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.556+0000 7fa99bf09180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T15:32:18.583 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.580+0000 7f8e16a12180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T15:32:18.611 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.612+0000 7fa99bf09180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T15:32:18.636 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.632+0000 7f8e16a12180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T15:32:18.643 INFO:teuthology.orchestra.run.smithi170.stdout:3 2024-09-21T15:32:18.660 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new f3780990-ed73-4f96-b93e-34e83e63d9e5 4 2024-09-21T15:32:18.666 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.664+0000 7fa99bf09180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T15:32:18.688 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.688+0000 7f8e16a12180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T15:32:18.746 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.744+0000 7fa99bf09180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T15:32:18.766 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.764+0000 7f8e16a12180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T15:32:18.801 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.800+0000 7fa99bf09180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T15:32:18.817 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.816+0000 7f8e16a12180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T15:32:18.862 INFO:teuthology.orchestra.run.smithi170.stdout:4 2024-09-21T15:32:18.876 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 36f2dd75-f924-443d-ac44-02c8d0fa6cc6 5 2024-09-21T15:32:18.880 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.880+0000 7fa99bf09180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T15:32:18.894 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.892+0000 7f8e16a12180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T15:32:18.934 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:18.932+0000 7fa99bf09180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T15:32:18.945 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:18.944+0000 7f8e16a12180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T15:32:19.079 INFO:teuthology.orchestra.run.smithi170.stdout:5 2024-09-21T15:32:19.090 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:19.088+0000 7fa99bf09180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T15:32:19.095 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new cae6719a-cf33-472a-b127-964ed06f3c5c 6 2024-09-21T15:32:19.098 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:19.096+0000 7f8e16a12180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T15:32:19.151 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:19.148+0000 7fa99bf09180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T15:32:19.157 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:19.156+0000 7f8e16a12180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T15:32:19.294 INFO:teuthology.orchestra.run.smithi170.stdout:6 2024-09-21T15:32:19.310 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new dde8b30b-cb83-4b0e-bdd9-b54ae839d666 7 2024-09-21T15:32:19.551 INFO:teuthology.orchestra.run.smithi170.stdout:7 2024-09-21T15:32:19.567 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T15:32:19.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T15:32:19.569 INFO:tasks.ceph.osd.0:Started 2024-09-21T15:32:19.569 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T15:32:19.569 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T15:32:19.571 INFO:tasks.ceph.osd.1:Started 2024-09-21T15:32:19.571 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T15:32:19.571 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T15:32:19.573 INFO:tasks.ceph.osd.2:Started 2024-09-21T15:32:19.574 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T15:32:19.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T15:32:19.575 INFO:tasks.ceph.osd.3:Started 2024-09-21T15:32:19.576 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T15:32:19.576 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T15:32:19.578 INFO:tasks.ceph.osd.4:Started 2024-09-21T15:32:19.578 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T15:32:19.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T15:32:19.579 INFO:tasks.ceph.osd.5:Started 2024-09-21T15:32:19.580 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T15:32:19.580 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T15:32:19.581 INFO:tasks.ceph.osd.6:Started 2024-09-21T15:32:19.582 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T15:32:19.582 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T15:32:19.586 INFO:tasks.ceph.osd.7:Started 2024-09-21T15:32:19.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:19.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:19.779 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:19.548914+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:19.781 INFO:tasks.ceph.mgr.x.smithi170.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T15:32:19.781 INFO:tasks.ceph.mgr.x.smithi170.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T15:32:19.781 INFO:tasks.ceph.mgr.x.smithi170.stderr: from numpy import show_config as show_numpy_config 2024-09-21T15:32:19.794 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T15:32:19.794 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T15:32:19.833 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:19.832+0000 7fa99bf09180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T15:32:19.847 INFO:tasks.ceph.mgr.y.smithi107.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-21T15:32:19.847 INFO:tasks.ceph.mgr.y.smithi107.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-21T15:32:19.847 INFO:tasks.ceph.mgr.y.smithi107.stderr: from numpy import show_config as show_numpy_config 2024-09-21T15:32:19.937 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:19.936+0000 7f8e16a12180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T15:32:19.965 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:19.964+0000 7fa99bf09180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T15:32:20.105 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:20.104+0000 7f8e16a12180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T15:32:20.123 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:20.120+0000 7fa99bf09180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T15:32:20.174 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:20.172+0000 7f84a4d107c0 -1 Falling back to public interface 2024-09-21T15:32:20.182 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:20.180+0000 7fdd8d4e07c0 -1 Falling back to public interface 2024-09-21T15:32:20.185 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:20.184+0000 7fa99bf09180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T15:32:20.189 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:20.188+0000 7fe902ae17c0 -1 Falling back to public interface 2024-09-21T15:32:20.194 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:20.192+0000 7f2969ab97c0 -1 Falling back to public interface 2024-09-21T15:32:20.205 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:20.204+0000 7f22e30427c0 -1 Falling back to public interface 2024-09-21T15:32:20.208 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:20.208+0000 7f0c92a4b7c0 -1 Falling back to public interface 2024-09-21T15:32:20.209 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:20.208+0000 7f6b0ec607c0 -1 Falling back to public interface 2024-09-21T15:32:20.213 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:20.212+0000 7f4f6579f7c0 -1 Falling back to public interface 2024-09-21T15:32:20.256 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:20.252+0000 7f8e16a12180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T15:32:20.323 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:20.320+0000 7fa99bf09180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T15:32:20.324 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:20.320+0000 7f8e16a12180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T15:32:20.465 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:20.464+0000 7f8e16a12180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T15:32:20.598 DEBUG:teuthology.orchestra.run.smithi107:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:20.773 INFO:teuthology.misc.health.smithi107.stdout: 2024-09-21T15:32:20.774 INFO:teuthology.misc.health.smithi107.stdout:{"epoch":10,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:19.548914+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:20.788 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T15:32:20.929 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:20.928+0000 7fa99bf09180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T15:32:21.003 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:21.000+0000 7fa99bf09180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T15:32:21.094 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.092+0000 7f8e16a12180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T15:32:21.169 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.168+0000 7f8e16a12180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T15:32:21.183 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:21.180+0000 7fa99bf09180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T15:32:21.256 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:21.256+0000 7fa99bf09180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T15:32:21.348 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.344+0000 7f8e16a12180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T15:32:21.421 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.420+0000 7f8e16a12180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T15:32:21.541 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:21.540+0000 7fa99bf09180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T15:32:21.602 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:21.600+0000 7fa99bf09180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T15:32:21.715 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.712+0000 7f8e16a12180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T15:32:21.776 INFO:tasks.ceph.mgr.y.smithi107.stderr:2024-09-21T15:32:21.772+0000 7f8e16a12180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T15:32:22.648 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:22.648+0000 7fa9434ba640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T15:32:22.651 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-09-21T15:32:22.648+0000 7fa9434ba640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T15:32:23.910 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:23.908+0000 7f2969ab97c0 -1 osd.4 0 log_to_monitors true 2024-09-21T15:32:23.981 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:23.980+0000 7fdd8d4e07c0 -1 osd.3 0 log_to_monitors true 2024-09-21T15:32:24.054 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:24.052+0000 7f0c92a4b7c0 -1 osd.7 0 log_to_monitors true 2024-09-21T15:32:24.079 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:24.076+0000 7fe902ae17c0 -1 osd.1 0 log_to_monitors true 2024-09-21T15:32:24.104 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:24.104+0000 7f22e30427c0 -1 osd.0 0 log_to_monitors true 2024-09-21T15:32:24.105 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:24.104+0000 7f84a4d107c0 -1 osd.2 0 log_to_monitors true 2024-09-21T15:32:24.149 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:24.148+0000 7f4f6579f7c0 -1 osd.5 0 log_to_monitors true 2024-09-21T15:32:24.153 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:24.152+0000 7f6b0ec607c0 -1 osd.6 0 log_to_monitors true 2024-09-21T15:32:27.387 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:27.384+0000 7f296523b640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T15:32:27.388 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:27.388+0000 7f4f6275c640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T15:32:27.406 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:27.404+0000 7f0c8e1cd640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T15:32:27.420 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:27.420+0000 7f4f59566640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:27.424 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:27.424+0000 7f295d880640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:27.435 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:27.432+0000 7f6b0a3e2640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T15:32:27.436 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:27.436+0000 7f0c86812640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:27.484 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:27.484+0000 7f6b02a27640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:27.595 DEBUG:teuthology.orchestra.run.smithi107:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:28.060 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:28.056+0000 7f84a0492640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T15:32:28.071 INFO:teuthology.misc.health.smithi107.stdout: 2024-09-21T15:32:28.071 INFO:teuthology.misc.health.smithi107.stdout:{"epoch":12,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:25.640399+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:28.072 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:28.072+0000 7fdd88c62640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T15:32:28.079 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:28.076+0000 7fe8ffa9e640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T15:32:28.094 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T15:32:28.094 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:28.092+0000 7f8498ad7640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:28.104 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:28.104+0000 7fe8f68a8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:28.117 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:28.116+0000 7fdd812a7640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:28.127 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:28.124+0000 7f22de7c4640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T15:32:28.157 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:28.156+0000 7f22d6e09640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:32:34.901 DEBUG:teuthology.orchestra.run.smithi107:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:35.194 INFO:teuthology.misc.health.smithi107.stdout: 2024-09-21T15:32:35.194 INFO:teuthology.misc.health.smithi107.stdout:{"epoch":16,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:31.720350+0000","last_up_change":"2024-09-21T15:32:28.438444+0000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6825","nonce":379929313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6827","nonce":379929313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6831","nonce":379929313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6829","nonce":379929313}]},"public_addr":"172.21.15.107:6825/379929313","cluster_addr":"172.21.15.107:6827/379929313","heartbeat_back_addr":"172.21.15.107:6831/379929313","heartbeat_front_addr":"172.21.15.107:6829/379929313","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:35.213 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T15:32:35.214 INFO:tasks.ceph:Creating RBD pool 2024-09-21T15:32:35.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T15:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'rbd' created 2024-09-21T15:32:35.790 DEBUG:teuthology.orchestra.run.smithi107:> rbd --cluster ceph pool init rbd 2024-09-21T15:32:38.823 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T15:32:38.823 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T15:32:38.823 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T15:32:38.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T15:32:38.824 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T15:32:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:39.189 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":4140676629},{"type":"v1","addr":"172.21.15.170:6833","nonce":4140676629}]},"active_addr":"172.21.15.170:6833/4140676629","active_change":"2024-09-21T15:32:21.622820+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3815078501}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2212408644}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3270855337}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3551288228}]}]} 2024-09-21T15:32:39.194 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T15:32:39.195 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T15:32:39.195 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:39.473 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:39.473 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":20,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:38.781880+0000","last_up_change":"2024-09-21T15:32:28.438444+0000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T15:32:35.528036+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6825","nonce":379929313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6827","nonce":379929313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6831","nonce":379929313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6829","nonce":379929313}]},"public_addr":"172.21.15.107:6825/379929313","cluster_addr":"172.21.15.107:6827/379929313","heartbeat_back_addr":"172.21.15.107:6831/379929313","heartbeat_front_addr":"172.21.15.107:6829/379929313","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:39.490 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T15:32:39.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:32:39.776 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:39.776 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":20,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:32:38.781880+0000","last_up_change":"2024-09-21T15:32:28.438444+0000","last_in_change":"2024-09-21T15:32:19.548914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T15:32:35.528036+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6825","nonce":379929313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6827","nonce":379929313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6831","nonce":379929313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":379929313},{"type":"v1","addr":"172.21.15.107:6829","nonce":379929313}]},"public_addr":"172.21.15.107:6825/379929313","cluster_addr":"172.21.15.107:6827/379929313","heartbeat_back_addr":"172.21.15.107:6831/379929313","heartbeat_front_addr":"172.21.15.107:6829/379929313","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:32:39.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T15:32:39.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T15:32:39.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T15:32:39.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T15:32:39.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T15:32:39.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T15:32:39.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T15:32:39.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T15:32:40.038 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:32:40.056 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:40.056 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:40.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:32:40.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:32:40.058 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:32:40.062 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:32:40.066 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:40.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:32:40.075 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.075 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:32:40.077 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:32:40.552 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.556 INFO:teuthology.orchestra.run.smithi107.stdout:55834574852 2024-09-21T15:32:40.570 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.578 INFO:teuthology.orchestra.run.smithi107.stdout:55834574852 2024-09-21T15:32:40.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-09-21T15:32:40.579 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:40.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-21T15:32:40.595 INFO:teuthology.orchestra.run.smithi107.stdout:55834574852 2024-09-21T15:32:40.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-09-21T15:32:40.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-09-21T15:32:40.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-09-21T15:32:40.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-09-21T15:32:40.625 INFO:teuthology.orchestra.run.smithi107.stdout:55834574852 2024-09-21T15:32:40.630 INFO:teuthology.orchestra.run.smithi107.stdout:55834574852 2024-09-21T15:32:40.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-09-21T15:32:40.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-09-21T15:32:41.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:32:41.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:32:41.601 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:32:41.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:32:41.612 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:32:41.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:32:41.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:32:41.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:32:42.107 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:42.121 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:42.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-09-21T15:32:42.134 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.134 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:42.135 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:42.141 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:42.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-09-21T15:32:42.151 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-09-21T15:32:42.159 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-09-21T15:32:42.167 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-09-21T15:32:42.173 INFO:teuthology.orchestra.run.smithi107.stdout:55834574854 2024-09-21T15:32:42.173 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.187 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:42.187 INFO:teuthology.orchestra.run.smithi107.stdout:55834574853 2024-09-21T15:32:42.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-21T15:32:42.190 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-09-21T15:32:42.205 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-09-21T15:32:42.206 DEBUG:teuthology.parallel:result is None 2024-09-21T15:32:42.206 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T15:32:42.206 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:32:42.514 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:42.515 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:32:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-21T15:32:41.639793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235312,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739408,"statfs":{"total":773094113280,"available":772853153792,"internally_reserved":0,"allocated":2797568,"data_stored":1714158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.874525"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.994311+0000","last_change":"2024-09-21T15:32:38.994707+0000","last_active":"2024-09-21T15:32:38.994311+0000","last_peered":"2024-09-21T15:32:38.994311+0000","last_clean":"2024-09-21T15:32:38.994311+0000","last_became_active":"2024-09-21T15:32:36.789346+0000","last_became_peered":"2024-09-21T15:32:36.789346+0000","last_unstale":"2024-09-21T15:32:38.994311+0000","last_undegraded":"2024-09-21T15:32:38.994311+0000","last_fullsized":"2024-09-21T15:32:38.994311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:58.785429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790013+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:32:38.790013+0000","last_peered":"2024-09-21T15:32:38.790013+0000","last_clean":"2024-09-21T15:32:38.790013+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:32:38.790013+0000","last_undegraded":"2024-09-21T15:32:38.790013+0000","last_fullsized":"2024-09-21T15:32:38.790013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776752+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:32:39.776752+0000","last_peered":"2024-09-21T15:32:39.776752+0000","last_clean":"2024-09-21T15:32:39.776752+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:32:39.776752+0000","last_undegraded":"2024-09-21T15:32:39.776752+0000","last_fullsized":"2024-09-21T15:32:39.776752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790456+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:32:38.790456+0000","last_peered":"2024-09-21T15:32:38.790456+0000","last_clean":"2024-09-21T15:32:38.790456+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:32:38.790456+0000","last_undegraded":"2024-09-21T15:32:38.790456+0000","last_fullsized":"2024-09-21T15:32:38.790456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.802783+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:32:38.802783+0000","last_peered":"2024-09-21T15:32:38.802783+0000","last_clean":"2024-09-21T15:32:38.802783+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:32:38.802783+0000","last_undegraded":"2024-09-21T15:32:38.802783+0000","last_fullsized":"2024-09-21T15:32:38.802783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.167795+0000","last_change":"2024-09-21T15:32:39.167953+0000","last_active":"2024-09-21T15:32:39.167795+0000","last_peered":"2024-09-21T15:32:39.167795+0000","last_clean":"2024-09-21T15:32:39.167795+0000","last_became_active":"2024-09-21T15:32:37.100650+0000","last_became_peered":"2024-09-21T15:32:37.100650+0000","last_unstale":"2024-09-21T15:32:39.167795+0000","last_undegraded":"2024-09-21T15:32:39.167795+0000","last_fullsized":"2024-09-21T15:32:39.167795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:54.344750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776646+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:32:39.776646+0000","last_peered":"2024-09-21T15:32:39.776646+0000","last_clean":"2024-09-21T15:32:39.776646+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:32:39.776646+0000","last_undegraded":"2024-09-21T15:32:39.776646+0000","last_fullsized":"2024-09-21T15:32:39.776646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.801164+0000","last_change":"2024-09-21T15:32:38.801778+0000","last_active":"2024-09-21T15:32:38.801164+0000","last_peered":"2024-09-21T15:32:38.801164+0000","last_clean":"2024-09-21T15:32:38.801164+0000","last_became_active":"2024-09-21T15:32:36.792972+0000","last_became_peered":"2024-09-21T15:32:36.792972+0000","last_unstale":"2024-09-21T15:32:38.801164+0000","last_undegraded":"2024-09-21T15:32:38.801164+0000","last_fullsized":"2024-09-21T15:32:38.801164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:23:48.925970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.564828+0000","last_change":"2024-09-21T15:32:30.735137+0000","last_active":"2024-09-21T15:32:38.564828+0000","last_peered":"2024-09-21T15:32:38.564828+0000","last_clean":"2024-09-21T15:32:38.564828+0000","last_became_active":"2024-09-21T15:32:30.732347+0000","last_became_peered":"2024-09-21T15:32:30.732347+0000","last_unstale":"2024-09-21T15:32:38.564828+0000","last_undegraded":"2024-09-21T15:32:38.564828+0000","last_fullsized":"2024-09-21T15:32:38.564828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:37.923876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:32:42.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:32:42.800 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:42.801 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:32:42.818 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-21T15:32:41.639793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235312,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739408,"statfs":{"total":773094113280,"available":772853153792,"internally_reserved":0,"allocated":2797568,"data_stored":1714158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.874525"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.994311+0000","last_change":"2024-09-21T15:32:38.994707+0000","last_active":"2024-09-21T15:32:38.994311+0000","last_peered":"2024-09-21T15:32:38.994311+0000","last_clean":"2024-09-21T15:32:38.994311+0000","last_became_active":"2024-09-21T15:32:36.789346+0000","last_became_peered":"2024-09-21T15:32:36.789346+0000","last_unstale":"2024-09-21T15:32:38.994311+0000","last_undegraded":"2024-09-21T15:32:38.994311+0000","last_fullsized":"2024-09-21T15:32:38.994311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:58.785429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790013+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:32:38.790013+0000","last_peered":"2024-09-21T15:32:38.790013+0000","last_clean":"2024-09-21T15:32:38.790013+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:32:38.790013+0000","last_undegraded":"2024-09-21T15:32:38.790013+0000","last_fullsized":"2024-09-21T15:32:38.790013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776752+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:32:39.776752+0000","last_peered":"2024-09-21T15:32:39.776752+0000","last_clean":"2024-09-21T15:32:39.776752+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:32:39.776752+0000","last_undegraded":"2024-09-21T15:32:39.776752+0000","last_fullsized":"2024-09-21T15:32:39.776752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790456+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:32:38.790456+0000","last_peered":"2024-09-21T15:32:38.790456+0000","last_clean":"2024-09-21T15:32:38.790456+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:32:38.790456+0000","last_undegraded":"2024-09-21T15:32:38.790456+0000","last_fullsized":"2024-09-21T15:32:38.790456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.802783+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:32:38.802783+0000","last_peered":"2024-09-21T15:32:38.802783+0000","last_clean":"2024-09-21T15:32:38.802783+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:32:38.802783+0000","last_undegraded":"2024-09-21T15:32:38.802783+0000","last_fullsized":"2024-09-21T15:32:38.802783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.167795+0000","last_change":"2024-09-21T15:32:39.167953+0000","last_active":"2024-09-21T15:32:39.167795+0000","last_peered":"2024-09-21T15:32:39.167795+0000","last_clean":"2024-09-21T15:32:39.167795+0000","last_became_active":"2024-09-21T15:32:37.100650+0000","last_became_peered":"2024-09-21T15:32:37.100650+0000","last_unstale":"2024-09-21T15:32:39.167795+0000","last_undegraded":"2024-09-21T15:32:39.167795+0000","last_fullsized":"2024-09-21T15:32:39.167795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:54.344750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776646+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:32:39.776646+0000","last_peered":"2024-09-21T15:32:39.776646+0000","last_clean":"2024-09-21T15:32:39.776646+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:32:39.776646+0000","last_undegraded":"2024-09-21T15:32:39.776646+0000","last_fullsized":"2024-09-21T15:32:39.776646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.801164+0000","last_change":"2024-09-21T15:32:38.801778+0000","last_active":"2024-09-21T15:32:38.801164+0000","last_peered":"2024-09-21T15:32:38.801164+0000","last_clean":"2024-09-21T15:32:38.801164+0000","last_became_active":"2024-09-21T15:32:36.792972+0000","last_became_peered":"2024-09-21T15:32:36.792972+0000","last_unstale":"2024-09-21T15:32:38.801164+0000","last_undegraded":"2024-09-21T15:32:38.801164+0000","last_fullsized":"2024-09-21T15:32:38.801164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:23:48.925970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.564828+0000","last_change":"2024-09-21T15:32:30.735137+0000","last_active":"2024-09-21T15:32:38.564828+0000","last_peered":"2024-09-21T15:32:38.564828+0000","last_clean":"2024-09-21T15:32:38.564828+0000","last_became_active":"2024-09-21T15:32:30.732347+0000","last_became_peered":"2024-09-21T15:32:30.732347+0000","last_unstale":"2024-09-21T15:32:38.564828+0000","last_undegraded":"2024-09-21T15:32:38.564828+0000","last_fullsized":"2024-09-21T15:32:38.564828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:37.923876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:32:42.819 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T15:32:42.819 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T15:32:42.819 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T15:32:42.819 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T15:32:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:43.113 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T15:32:43.130 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T15:32:43.131 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T15:32:43.140 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T15:32:43.140 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}} 2024-09-21T15:32:43.140 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T15:32:43.141 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T15:32:43.443 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T15:32:43.443 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T15:32:43.443 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:32:43.720 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:43.721 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:32:43.737 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T15:32:43.640408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235312,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739408,"statfs":{"total":773094113280,"available":772853153792,"internally_reserved":0,"allocated":2797568,"data_stored":1714158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.862349"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.994311+0000","last_change":"2024-09-21T15:32:38.994707+0000","last_active":"2024-09-21T15:32:38.994311+0000","last_peered":"2024-09-21T15:32:38.994311+0000","last_clean":"2024-09-21T15:32:38.994311+0000","last_became_active":"2024-09-21T15:32:36.789346+0000","last_became_peered":"2024-09-21T15:32:36.789346+0000","last_unstale":"2024-09-21T15:32:38.994311+0000","last_undegraded":"2024-09-21T15:32:38.994311+0000","last_fullsized":"2024-09-21T15:32:38.994311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:58.785429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790013+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:32:38.790013+0000","last_peered":"2024-09-21T15:32:38.790013+0000","last_clean":"2024-09-21T15:32:38.790013+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:32:38.790013+0000","last_undegraded":"2024-09-21T15:32:38.790013+0000","last_fullsized":"2024-09-21T15:32:38.790013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776752+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:32:39.776752+0000","last_peered":"2024-09-21T15:32:39.776752+0000","last_clean":"2024-09-21T15:32:39.776752+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:32:39.776752+0000","last_undegraded":"2024-09-21T15:32:39.776752+0000","last_fullsized":"2024-09-21T15:32:39.776752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790456+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:32:38.790456+0000","last_peered":"2024-09-21T15:32:38.790456+0000","last_clean":"2024-09-21T15:32:38.790456+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:32:38.790456+0000","last_undegraded":"2024-09-21T15:32:38.790456+0000","last_fullsized":"2024-09-21T15:32:38.790456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.802783+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:32:38.802783+0000","last_peered":"2024-09-21T15:32:38.802783+0000","last_clean":"2024-09-21T15:32:38.802783+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:32:38.802783+0000","last_undegraded":"2024-09-21T15:32:38.802783+0000","last_fullsized":"2024-09-21T15:32:38.802783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.167795+0000","last_change":"2024-09-21T15:32:39.167953+0000","last_active":"2024-09-21T15:32:39.167795+0000","last_peered":"2024-09-21T15:32:39.167795+0000","last_clean":"2024-09-21T15:32:39.167795+0000","last_became_active":"2024-09-21T15:32:37.100650+0000","last_became_peered":"2024-09-21T15:32:37.100650+0000","last_unstale":"2024-09-21T15:32:39.167795+0000","last_undegraded":"2024-09-21T15:32:39.167795+0000","last_fullsized":"2024-09-21T15:32:39.167795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:54.344750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776646+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:32:39.776646+0000","last_peered":"2024-09-21T15:32:39.776646+0000","last_clean":"2024-09-21T15:32:39.776646+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:32:39.776646+0000","last_undegraded":"2024-09-21T15:32:39.776646+0000","last_fullsized":"2024-09-21T15:32:39.776646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.801164+0000","last_change":"2024-09-21T15:32:38.801778+0000","last_active":"2024-09-21T15:32:38.801164+0000","last_peered":"2024-09-21T15:32:38.801164+0000","last_clean":"2024-09-21T15:32:38.801164+0000","last_became_active":"2024-09-21T15:32:36.792972+0000","last_became_peered":"2024-09-21T15:32:36.792972+0000","last_unstale":"2024-09-21T15:32:38.801164+0000","last_undegraded":"2024-09-21T15:32:38.801164+0000","last_fullsized":"2024-09-21T15:32:38.801164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:23:48.925970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.564828+0000","last_change":"2024-09-21T15:32:30.735137+0000","last_active":"2024-09-21T15:32:38.564828+0000","last_peered":"2024-09-21T15:32:38.564828+0000","last_clean":"2024-09-21T15:32:38.564828+0000","last_became_active":"2024-09-21T15:32:30.732347+0000","last_became_peered":"2024-09-21T15:32:30.732347+0000","last_unstale":"2024-09-21T15:32:38.564828+0000","last_undegraded":"2024-09-21T15:32:38.564828+0000","last_fullsized":"2024-09-21T15:32:38.564828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:37.923876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:32:43.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:32:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:44.014 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:32:44.029 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T15:32:43.640408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":235312,"kb_used_data":2732,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739408,"statfs":{"total":773094113280,"available":772853153792,"internally_reserved":0,"allocated":2797568,"data_stored":1714158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.862349"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.994311+0000","last_change":"2024-09-21T15:32:38.994707+0000","last_active":"2024-09-21T15:32:38.994311+0000","last_peered":"2024-09-21T15:32:38.994311+0000","last_clean":"2024-09-21T15:32:38.994311+0000","last_became_active":"2024-09-21T15:32:36.789346+0000","last_became_peered":"2024-09-21T15:32:36.789346+0000","last_unstale":"2024-09-21T15:32:38.994311+0000","last_undegraded":"2024-09-21T15:32:38.994311+0000","last_fullsized":"2024-09-21T15:32:38.994311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:58.785429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790013+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:32:38.790013+0000","last_peered":"2024-09-21T15:32:38.790013+0000","last_clean":"2024-09-21T15:32:38.790013+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:32:38.790013+0000","last_undegraded":"2024-09-21T15:32:38.790013+0000","last_fullsized":"2024-09-21T15:32:38.790013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776752+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:32:39.776752+0000","last_peered":"2024-09-21T15:32:39.776752+0000","last_clean":"2024-09-21T15:32:39.776752+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:32:39.776752+0000","last_undegraded":"2024-09-21T15:32:39.776752+0000","last_fullsized":"2024-09-21T15:32:39.776752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.790456+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:32:38.790456+0000","last_peered":"2024-09-21T15:32:38.790456+0000","last_clean":"2024-09-21T15:32:38.790456+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:32:38.790456+0000","last_undegraded":"2024-09-21T15:32:38.790456+0000","last_fullsized":"2024-09-21T15:32:38.790456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.802783+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:32:38.802783+0000","last_peered":"2024-09-21T15:32:38.802783+0000","last_clean":"2024-09-21T15:32:38.802783+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:32:38.802783+0000","last_undegraded":"2024-09-21T15:32:38.802783+0000","last_fullsized":"2024-09-21T15:32:38.802783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.167795+0000","last_change":"2024-09-21T15:32:39.167953+0000","last_active":"2024-09-21T15:32:39.167795+0000","last_peered":"2024-09-21T15:32:39.167795+0000","last_clean":"2024-09-21T15:32:39.167795+0000","last_became_active":"2024-09-21T15:32:37.100650+0000","last_became_peered":"2024-09-21T15:32:37.100650+0000","last_unstale":"2024-09-21T15:32:39.167795+0000","last_undegraded":"2024-09-21T15:32:39.167795+0000","last_fullsized":"2024-09-21T15:32:39.167795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:54.344750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:39.776646+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:32:39.776646+0000","last_peered":"2024-09-21T15:32:39.776646+0000","last_clean":"2024-09-21T15:32:39.776646+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:32:39.776646+0000","last_undegraded":"2024-09-21T15:32:39.776646+0000","last_fullsized":"2024-09-21T15:32:39.776646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.801164+0000","last_change":"2024-09-21T15:32:38.801778+0000","last_active":"2024-09-21T15:32:38.801164+0000","last_peered":"2024-09-21T15:32:38.801164+0000","last_clean":"2024-09-21T15:32:38.801164+0000","last_became_active":"2024-09-21T15:32:36.792972+0000","last_became_peered":"2024-09-21T15:32:36.792972+0000","last_unstale":"2024-09-21T15:32:38.801164+0000","last_undegraded":"2024-09-21T15:32:38.801164+0000","last_fullsized":"2024-09-21T15:32:38.801164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:23:48.925970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T15:32:38.564828+0000","last_change":"2024-09-21T15:32:30.735137+0000","last_active":"2024-09-21T15:32:38.564828+0000","last_peered":"2024-09-21T15:32:38.564828+0000","last_clean":"2024-09-21T15:32:38.564828+0000","last_became_active":"2024-09-21T15:32:30.732347+0000","last_became_peered":"2024-09-21T15:32:30.732347+0000","last_unstale":"2024-09-21T15:32:38.564828+0000","last_undegraded":"2024-09-21T15:32:38.564828+0000","last_fullsized":"2024-09-21T15:32:38.564828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:36:37.923876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":786432,"data_stored":653396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":68860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":63028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":68879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":659228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:32:44.031 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T15:32:44.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T15:32:44.308 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 20 2024-09-21T15:32:44.308 INFO:teuthology.orchestra.run.smithi107.stdout:fsid 09582c7a-ec34-4064-810b-b65e59e21b91 2024-09-21T15:32:44.308 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-09-21T15:32:16.991419+0000 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-09-21T15:32:38.781880+0000 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 4 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client jewel 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release reef 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 20 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout: removed_snaps_queue [2~1] 2024-09-21T15:32:44.309 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 8 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/379929313,v1:172.21.15.107:6825/379929313] [v2:172.21.15.107:6826/379929313,v1:172.21.15.107:6827/379929313] exists,up d2f9d147-bbb0-470e-8441-d891316be3ee 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/615179745,v1:172.21.15.107:6817/615179745] [v2:172.21.15.107:6818/615179745,v1:172.21.15.107:6819/615179745] exists,up e13f399d-5ddd-47e1-879e-eda2f7ec930f 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/3712995483,v1:172.21.15.107:6801/3712995483] [v2:172.21.15.107:6802/3712995483,v1:172.21.15.107:6803/3712995483] exists,up 84d52d47-67ab-43ca-a983-5568d9cf1975 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/3212640731,v1:172.21.15.107:6809/3212640731] [v2:172.21.15.107:6810/3212640731,v1:172.21.15.107:6811/3212640731] exists,up adbb6235-d8c3-4b20-a7e4-0046d22ed5fc 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/1739937222,v1:172.21.15.170:6801/1739937222] [v2:172.21.15.170:6802/1739937222,v1:172.21.15.170:6803/1739937222] exists,up f3780990-ed73-4f96-b93e-34e83e63d9e5 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6822/4269648595,v1:172.21.15.170:6824/4269648595] [v2:172.21.15.170:6825/4269648595,v1:172.21.15.170:6827/4269648595] exists,up 36f2dd75-f924-443d-ac44-02c8d0fa6cc6 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6809/3971003516,v1:172.21.15.170:6810/3971003516] [v2:172.21.15.170:6812/3971003516,v1:172.21.15.170:6813/3971003516] exists,up cae6719a-cf33-472a-b127-964ed06f3c5c 2024-09-21T15:32:44.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/3076528025,v1:172.21.15.170:6811/3076528025] [v2:172.21.15.170:6815/3076528025,v1:172.21.15.170:6818/3076528025] exists,up dde8b30b-cb83-4b0e-bdd9-b54ae839d666 2024-09-21T15:32:44.325 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/379929313,v1:172.21.15.107:6825/379929313] [v2:172.21.15.107:6826/379929313,v1:172.21.15.107:6827/379929313] exists,up d2f9d147-bbb0-470e-8441-d891316be3ee', 'osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/615179745,v1:172.21.15.107:6817/615179745] [v2:172.21.15.107:6818/615179745,v1:172.21.15.107:6819/615179745] exists,up e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/3712995483,v1:172.21.15.107:6801/3712995483] [v2:172.21.15.107:6802/3712995483,v1:172.21.15.107:6803/3712995483] exists,up 84d52d47-67ab-43ca-a983-5568d9cf1975', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/3212640731,v1:172.21.15.107:6809/3212640731] [v2:172.21.15.107:6810/3212640731,v1:172.21.15.107:6811/3212640731] exists,up adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/1739937222,v1:172.21.15.170:6801/1739937222] [v2:172.21.15.170:6802/1739937222,v1:172.21.15.170:6803/1739937222] exists,up f3780990-ed73-4f96-b93e-34e83e63d9e5', 'osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6822/4269648595,v1:172.21.15.170:6824/4269648595] [v2:172.21.15.170:6825/4269648595,v1:172.21.15.170:6827/4269648595] exists,up 36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'osd.6 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6809/3971003516,v1:172.21.15.170:6810/3971003516] [v2:172.21.15.170:6812/3971003516,v1:172.21.15.170:6813/3971003516] exists,up cae6719a-cf33-472a-b127-964ed06f3c5c', 'osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/3076528025,v1:172.21.15.170:6811/3076528025] [v2:172.21.15.170:6815/3076528025,v1:172.21.15.170:6818/3076528025] exists,up dde8b30b-cb83-4b0e-bdd9-b54ae839d666'] 2024-09-21T15:32:44.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T15:32:44.439 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mon.a", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster": "ceph", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "admin_socket_mode": "", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_client_required": "cephx, none", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_cluster_required": "cephx", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_debug": "true", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T15:32:44.440 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_service_required": "cephx", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "auth_supported": "cephx", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_aio": "true", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_aio_reap_max": "16", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_async_discard": "false", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_block_size": "4096", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_debug_aio": "false", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T15:32:44.441 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_enable_discard": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_flock_retry": "3", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_inject_crash": "0", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_ioring": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_ioring_hipri": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T15:32:44.442 INFO:teuthology.orchestra.run.smithi107.stdout: "bdev_type": "", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_allocator": "hybrid", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_buffered_io": "true", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T15:32:44.443 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_replay_recovery": "false", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_sync_write": "false", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T15:32:44.444 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_allocator": "hybrid", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_create": "true", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_db_create": "false", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_db_path": "", 2024-09-21T15:32:44.445 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_db_size": "0", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_path": "", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_size": "107374182400", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_wal_create": "false", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_wal_path": "", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bluefs": "true", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T15:32:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_autotune": "true", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_size": "0", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T15:32:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_type": "2q", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_clone_cow": "true", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T15:32:44.448 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_mode": "none", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_freelist": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T15:32:44.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T15:32:44.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fail_eio": "false", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T15:32:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T15:32:44.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_blob_size": "0", 2024-09-21T15:32:44.453 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T15:32:44.454 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_spdk_mem": "512", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T15:32:44.455 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_tracing": "false", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T15:32:44.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T15:32:44.457 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_cluster_require_version": "2", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_require_signatures": "false", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_require_version": "2", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_service_require_signatures": "false", 2024-09-21T15:32:44.458 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_service_require_version": "2", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "cephx_sign_messages": "true", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "chdir": "", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_acl_type": "", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_asio_thread_count": "2", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_cache_mid": "0.750000", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_cache_size": "16384", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_caps_release_delay": "5", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_check_pool_perm": "true", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_debug_force_sync_read": "false", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_debug_getattr_caps": "false", 2024-09-21T15:32:44.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_debug_inject_features": "", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_die_on_failed_remount": "false", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_dirsize_rbytes": "true", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_force_lazyio": "false", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_fs": "", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_inject_release_failure": "false", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_max_inline_size": "4096", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_mds_namespace": "", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_metadata": "", 2024-09-21T15:32:44.460 INFO:teuthology.orchestra.run.smithi107.stdout: "client_mount_gid": "-1", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_mount_timeout": "300", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_mount_uid": "-1", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_mountpoint": "/", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_notify_timeout": "10", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc": "true", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc_max_objects": "1000", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc_size": "209715200", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_permissions": "true", 2024-09-21T15:32:44.461 INFO:teuthology.orchestra.run.smithi107.stdout: "client_quota": "true", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_quota_df": "true", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_readahead_max_bytes": "0", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_readahead_max_periods": "4", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_readahead_min": "131072", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_reconnect_stale": "false", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_shutdown_timeout": "30", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_snapdir": ".snap", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_tick_interval": "1", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_trace": "", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T15:32:44.462 INFO:teuthology.orchestra.run.smithi107.stdout: "client_use_faked_inos": "false", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "client_use_random_mds": "false", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_graylog": "false", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_graylog_port": "12201", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_monitors": "default=true", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_syslog": "false", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "clog_to_syslog_level": "info", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "-", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_network": "", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_network_interface": "", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "compressor_zlib_isal": "false", 2024-09-21T15:32:44.463 INFO:teuthology.orchestra.run.smithi107.stdout: "compressor_zlib_level": "5", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "compressor_zstd_level": "1", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crimson_seastar_smp": "1", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_location": "", 2024-09-21T15:32:44.464 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_location_hook": "", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_location_hook_timeout": "10", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "daemonize": "false", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "daos_pool": "tank", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_alienstore": "0/5", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_asok": "1/5", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_asok_assert_abort": "false", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T15:32:44.465 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_auth": "20/20", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_bdev": "1/3", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_bluefs": "1/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_bluestore": "1/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_buffer": "0/1", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_cephsqlite": "0/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_client": "0/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_compressor": "1/5", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_context": "0/1", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_crush": "1/1", 2024-09-21T15:32:44.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_crypto": "1/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_cyanstore": "0/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_dpdk": "1/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_eventtrace": "1/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_filer": "0/1", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_filestore": "1/3", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_finisher": "1/1", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_fuse": "1/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T15:32:44.467 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_javaclient": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_journal": "1/3", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_journaler": "0/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_kstore": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_leveldb": "4/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_lockdep": "0/1", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mclock": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds_balancer": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds_locker": "1/5", 2024-09-21T15:32:44.468 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds_log": "1/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mds_migrator": "1/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_memstore": "1/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mgr": "2/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mgrc": "1/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_mon": "20/20", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_monc": "0/10", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_ms": "1/1", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_none": "0/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_objclass": "0/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_objectcacher": "0/5", 2024-09-21T15:32:44.469 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_objecter": "0/1", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_optracker": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_osd": "1/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_paxos": "20/20", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_perfcounter": "1/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_prioritycache": "1/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rados": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rbd": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rbd_replay": "0/5", 2024-09-21T15:32:44.470 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_refs": "0/0", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_reserver": "1/1", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw_access": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw_flight": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rgw_sync": "1/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_rocksdb": "4/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore": "0/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_backref": "0/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_cache": "0/5", 2024-09-21T15:32:44.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_device": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_epm": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_journal": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_lba": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_odata": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_omap": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_onode": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_t": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_seastore_tm": "0/5", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_striper": "0/1", 2024-09-21T15:32:44.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_test": "0/5", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_throttle": "1/1", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_timer": "0/1", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "debug_tp": "0/5", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "device_failure_prediction_mode": "none", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "err_to_graylog": "false", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "err_to_journald": "false", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "err_to_stderr": "true", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "err_to_syslog": "false", 2024-09-21T15:32:44.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event_tracing": "false", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_http_port": "9926", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_prio_limit": "5", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_sort_metrics": "true", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "exporter_stats_period": "5", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "fake_statfs_for_testing": "0", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "fatal_signal_handlers": "true", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "filer_max_purge_ops": "10", 2024-09-21T15:32:44.474 INFO:teuthology.orchestra.run.smithi107.stdout: "filer_max_truncate_ops": "128", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_blackhole": "false", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_btrfs_snap": "true", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_caller_concurrency": "10", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_debug_omap_check": "false", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_debug_verify_split": "false", 2024-09-21T15:32:44.475 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_dump_file": "", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fadvise": "true", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fail_eio": "true", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fd_cache_size": "128", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fiemap": "false", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T15:32:44.476 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_inject_stall": "0", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_journal_parallel": "false", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_journal_trailing": "false", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_journal_writeahead": "false", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_kill_at": "0", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T15:32:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_merge_threshold": "-10", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_odsync_write": "false", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_omap_backend_path": "", 2024-09-21T15:32:44.478 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_op_threads": "2", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_punch_hole": "false", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T15:32:44.479 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_queue_max_ops": "50", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_seek_data_hole": "false", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_sloppy_crc": "false", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_splice": "false", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_split_multiple": "2", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_split_rand_factor": "20", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_update_to": "1000", 2024-09-21T15:32:44.480 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T15:32:44.481 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_xfs_extsize": "false", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "filestore_zfs_snap": "false", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fio_dir": "/tmp/fio", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_allow_other": "true", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_big_writes": "true", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_debug": "false", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_default_permissions": "false", 2024-09-21T15:32:44.482 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_disable_pagecache": "false", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_max_write": "0", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_multithreaded": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_require_active_mds": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_set_user_groups": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_splice_move": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_splice_read": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_splice_write": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T15:32:44.483 INFO:teuthology.orchestra.run.smithi107.stdout: "gss_target_name": "ceph", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_file": "", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_inject_failure": "0", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_interval": "5", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "host": "smithi107", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T15:32:44.484 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "inject_early_sigterm": "false", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "jaeger_agent_port": "6799", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "jaeger_tracing_enable": "false", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_aio": "true", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_align_min_size": "65536", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_block_align": "true", 2024-09-21T15:32:44.485 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_block_size": "4096", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_dio": "true", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_discard": "false", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_force_aio": "false", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_ignore_corruption": "false", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_max_write_entries": "100", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_replay_from": "0", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T15:32:44.486 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_write_header_frequency": "0", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "journal_zero_on_create": "false", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "journaler_prefetch_periods": "10", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "journaler_prezero_periods": "5", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "journaler_write_head_interval": "15", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "key": "", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "keyfile": "", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_backend": "rocksdb", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T15:32:44.487 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_max_bytes": "67108864", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_max_ops": "512", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_onode_map_size": "1024", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "kstore_sync_transaction": "false", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_block_size": "65536", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_bloom_size": "0", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_cache_size": "536870912", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_compact_on_mount": "false", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_compression": "false", 2024-09-21T15:32:44.488 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_log": "", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_max_open_files": "0", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_paranoid": "false", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "librados_thread_count": "2", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "lockdep": "false", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "lockdep_force_backtrace": "false", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_coarse_timestamps": "true", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_flush_on_exit": "false", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T15:32:44.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_graylog_port": "12201", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_max_new": "1000", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_max_recent": "10000", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_stderr_prefix": "", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": "true", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_graylog": "false", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_journald": "false", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_stderr": "false", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_syslog": "false", 2024-09-21T15:32:44.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_action_on_write_error": "1", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_alternate_name_max": "8192", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_asio_thread_count": "2", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_export_pin": "true", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T15:32:44.491 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_interval": "10", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_max": "-1", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_max_until": "-1", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_merge_size": "50", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_mode": "0", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T15:32:44.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_split_bits": "3", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_split_size": "10000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_mid": "0.700000", 2024-09-21T15:32:44.493 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_trim_interval": "1", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T15:32:44.494 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_debug_auth_pins": "false", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_debug_frag": "false", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_debug_scatterstat": "false", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_debug_subtrees": "false", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_default_dir_hash": "2", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_defer_session_stale": "true", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T15:32:44.495 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dir_max_entries": "0", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dir_prefetch": "true", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_early_reply": "true", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_enable_op_tracker": "true", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_enforce_unique_name": "true", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T15:32:44.496 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T15:32:44.497 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_health_dummy": "false", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_join_fs": "", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_journal_format": "1", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_create_at": "0", 2024-09-21T15:32:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_export_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_import_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_journal_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_link_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_openc_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_kill_rename_at": "0", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_max_events": "-1", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_max_segments": "128", 2024-09-21T15:32:44.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_pause": "false", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_segment_size": "0", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_completed_requests": "100000", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_export_size": "20971520", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_file_recover": "32", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_purge_files": "64", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_purge_ops": "8192", 2024-09-21T15:32:44.500 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_metrics_update_interval": "2", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_min_caps_per_client": "100", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_numa_node": "-1", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_history_duration": "600", 2024-09-21T15:32:44.501 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_history_size": "20", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_op_log_threshold": "5", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_ping_grace": "15", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_ping_interval": "5", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_max_caps": "30000", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T15:32:44.502 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_replay_interval": "1.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_root_ino_gid": "0", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_root_ino_uid": "0", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T15:32:44.503 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_shutdown_check": "0", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_skip_ino": "0", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_snap_min_uid": "0", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_snap_rstat": "false", 2024-09-21T15:32:44.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_symlink_recovery": "true", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_thrash_exports": "0", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_thrash_fragments": "0", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_tick_interval": "5.000000", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_valgrind_exit": "false", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_verify_backtrace": "1", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_verify_scatter": "false", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T15:32:44.505 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_wipe_sessions": "false", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool_debug": "false", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "memstore_page_set": "false", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "memstore_page_size": "65536", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_client_bytes": "134217728", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_client_messages": "512", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T15:32:44.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_disabled_modules": "", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_mds_messages": "128", 2024-09-21T15:32:44.507 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_mon_messages": "128", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_osd_messages": "8192", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_pool": "true", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_standby_modules": "true", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_stats_period": "5", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_stats_threshold": "5", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_tick_period": "2", 2024-09-21T15:32:44.508 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_allow_pool_delete": "true", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_bytes": "104857600", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T15:32:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T15:32:44.510 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_compact_on_start": "false", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_compact_on_trim": "true", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T15:32:44.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_cpu_threads": "4", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_data_avail_crit": "5", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_data_avail_warn": "5", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_dump_json": "false", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T15:32:44.512 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_extra_checks": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_no_require_quincy": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_election_default_strategy": "1", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_election_timeout": "5.000000", 2024-09-21T15:32:44.513 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_enable_op_tracker": "true", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_fake_pool_delete": "false", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_force_quorum_join": "false", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_log_update_period": "5", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_max_detail": "50", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_to_clog": "true", 2024-09-21T15:32:44.514 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_host": "172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790],172.21.15.170", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_host_override": "", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_initial_members": "", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_lease": "5.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T15:32:44.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_log_full_interval": "50", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_log_max": "10000", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_log_max_summary": "50", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_log_epochs": "500", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_osd": "10000", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T15:32:44.516 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_memory_autotune": "true", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_memory_target": "2147483648", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_digest_period": "5", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_complaint_time": "30", 2024-09-21T15:32:44.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_history_duration": "600", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_history_size": "20", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_op_log_threshold": "5", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T15:32:44.518 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_cache_size": "500", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T15:32:44.519 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T15:32:44.520 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_report_timeout": "900", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T15:32:44.521 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T15:32:44.522 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_reweight_max_osds": "4", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_scrub_interval": "86400", 2024-09-21T15:32:44.523 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_scrub_max_keys": "100", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_scrub_timeout": "300", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_session_timeout": "300", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_smart_report_timeout": "5", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_stretch_pool_size": "4", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T15:32:44.524 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_debug": "false", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_tick_interval": "5", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T15:32:44.525 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T15:32:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_older_version": "true", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "monmap": "", 2024-09-21T15:32:44.527 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "motr_tracing_enabled": "false", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_op_threads": "3", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_cm": "false", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_device_name": "", 2024-09-21T15:32:44.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T15:32:44.529 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_sl": "3", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_rdma_type": "ib", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_async_reap_threshold": "5", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_before_connect": "false", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_ipv4": "true", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_ipv6": "false", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_msgr1": "true", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_msgr2": "true", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_port_max": "7568", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_port_min": "6800", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T15:32:44.530 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_retry_count": "3", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_bind_retry_delay": "5", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_blackhole_client": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_blackhole_mds": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_blackhole_mgr": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_blackhole_mon": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_blackhole_osd": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_client_mode": "crc secure", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_cluster_type": "", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_compress_secure": "false", 2024-09-21T15:32:44.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_crc_data": "true", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_crc_header": "true", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_die_on_bug": "true", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_die_on_old_message": "true", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T15:32:44.532 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_hugepages": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_lro": "true", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_pmd": "", 2024-09-21T15:32:44.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_port_id": "0", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_dump_on_send": "false", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_delay_type": "", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_network_congestion": "0", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T15:32:44.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_max_accept_failures": "4", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_max_backoff": "15.000000", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_osd_compress_mode": "none", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_pq_min_cost": "65536", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_public_type": "", 2024-09-21T15:32:44.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_service_mode": "crc secure", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_tcp_nodelay": "true", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "ms_type": "async+posix", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "no_config_file": "false", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_inflight_ops": "1024", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T15:32:44.536 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter_timeout": "10.000000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "objectstore_blackhole": "false", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "openssl_engine_opts": "", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_max_ops": "4", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T15:32:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_agent_slop": "0.020000", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_asio_thread_count": "2", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backfill_scan_max": "512", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backfill_scan_min": "64", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backoff_on_peering": "false", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T15:32:44.538 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_beacon_report_interval": "300", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bench_duration": "30", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_blkin_trace_all": "false", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T15:32:44.539 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_class_update_on_start": "true", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_client_message_cap": "256", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_client_op_priority": "63", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_client_watch_timeout": "30", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_command_max_records": "256", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_compact_on_start": "false", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T15:32:44.540 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_crush_update_on_start": "true", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T15:32:44.541 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_op_order": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_shutdown": "false", 2024-09-21T15:32:44.542 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T15:32:44.543 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_default_notify_timeout": "30", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_enable_op_tracker": "true", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T15:32:44.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_fast_info": "true", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_fast_shutdown": "true", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_function_tracing": "false", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_grace": "20", 2024-09-21T15:32:44.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_interval": "6", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_stale": "600", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T15:32:44.546 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_journal_size": "5120", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_kill_backfill_at": "0", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_map_cache_size": "50", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_map_dedup": "true", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_map_message_max": "40", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T15:32:44.547 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_attr_name_len": "100", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_attr_size": "0", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_backfills": "1", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_markdown_count": "5", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_markdown_period": "600", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_object_name_len": "2048", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_object_size": "134217728", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T15:32:44.548 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_pgls": "1024", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_push_cost": "8388608", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_push_objects": "10", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_scrubs": "3", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_max_write_size": "90", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T15:32:44.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_profile": "balanced", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T15:32:44.550 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_base": "805306368", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_target": "4294967296", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_target_autotune": "false", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T15:32:44.551 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_min_recovery_priority": "0", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mon_report_interval": "5", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_num_cache_shards": "32", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_numa_node": "-1", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T15:32:44.552 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_objecter_finishers": "1", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_objectstore": "bluestore", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_objectstore_fuse": "false", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_objectstore_tracing": "false", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_history_duration": "600", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_history_size": "20", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_log_threshold": "5", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_shards": "0", 2024-09-21T15:32:44.553 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_op_thread_timeout": "15", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_open_classes_on_start": "true", 2024-09-21T15:32:44.554 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_os_flags": "0", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_peering_op_priority": "255", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_delete_priority": "5", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-21T15:32:44.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_crimson": "false", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T15:32:44.556 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_flags": "0", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_min_size": "0", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_size": "2", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_default_type": "replicated", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T15:32:44.557 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_cost": "20971520", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_active": "0", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T15:32:44.558 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_op_priority": "3", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_priority": "5", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_repair_during_recovery": "false", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T15:32:44.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T15:32:44.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_cost": "52428800", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_end_hour": "0", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T15:32:44.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_priority": "5", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_skip_data_digest": "false", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_smart_report_timeout": "5", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T15:32:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_priority": "5", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_target_transaction_size": "30", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T15:32:44.563 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_tracing": "false", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_use_stale_snap": "false", 2024-09-21T15:32:44.564 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_kill_at": "0", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_max_join_drift": "10", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_min": "500", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_min_wait": "0.050000", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_service_trim_max": "500", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_service_trim_min": "250", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_stash_full_interval": "25", 2024-09-21T15:32:44.565 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_trim_max": "500", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "paxos_trim_min": "250", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "perf": "true", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "-", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrv": "", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_bind_addr": "-", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_network": "", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_network_interface": "", 2024-09-21T15:32:44.566 INFO:teuthology.orchestra.run.smithi107.stdout: "qat_compressor_enabled": "false", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rados_mon_op_timeout": "0", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rados_osd_op_timeout": "0", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rados_tracing": "false", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_atime_update_interval": "60", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T15:32:44.567 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache": "true", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_policy": "writearound", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_size": "33554432", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_compression_hint": "none", 2024-09-21T15:32:44.568 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_clone_format": "auto", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_data_pool": "", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_features": "61", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_format": "2", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_map_options": "", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_order": "22", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_pool": "rbd", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_stripe_count": "0", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T15:32:44.569 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_io_scheduler": "simple", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T15:32:44.570 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_order": "24", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_pool": "", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_journal_splay_width": "4", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T15:32:44.571 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T15:32:44.572 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T15:32:44.573 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_op_threads": "1", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_plugins": "", 2024-09-21T15:32:44.574 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T15:32:44.575 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T15:32:44.576 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_tracing": "false", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_validate_names": "true", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_validate_pool": "true", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "restapi_base_url": "", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "restapi_log_level": "", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_admin_entry": "admin", 2024-09-21T15:32:44.577 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_backend_store": "rados", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_barbican_url": "", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_beast_enable_async": "true", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T15:32:44.578 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_cache_enabled": "true", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_cache_lru_size": "10000", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_config_store": "rados", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_content_length_compat": "false", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_cross_domain_policy": "", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T15:32:44.579 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T15:32:44.580 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T15:32:44.581 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T15:32:44.582 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T15:32:44.583 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T15:32:44.584 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T15:32:44.584 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_log_window": "30", 2024-09-21T15:32:44.584 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T15:32:44.584 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T15:32:44.585 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T15:32:44.585 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T15:32:44.585 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T15:32:44.585 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T15:32:44.586 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dns_name": "", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dns_s3website_name": "", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T15:32:44.587 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_ops_log": "false", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_static_website": "false", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enable_usage_log": "false", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_expose_bucket": "false", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_extended_http_attrs": "", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_filter": "none", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T15:32:44.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_objs": "32", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T15:32:44.589 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_init_timeout": "300", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_password": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_project": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T15:32:44.590 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_token": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_admin_user": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_api_version": "2", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T15:32:44.591 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_url": "", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_max_objs": "32", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_max_worker": "3", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T15:32:44.592 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_log_http_headers": "", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-21T15:32:44.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_attr_size": "0", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_listing_results": "1000", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_notify_retries": "10", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T15:32:44.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T15:32:44.595 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T15:32:44.596 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_num_control_oids": "8", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_numa_node": "-1", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_op_tracing": "false", 2024-09-21T15:32:44.597 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_opa_token": "", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_opa_url": "", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ops_log_rados": "false", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T15:32:44.598 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_print_continue": "true", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_rados_tracing": "false", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_realm": "", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_realm_id": "", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T15:32:44.599 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_region": "", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_request_uri": "", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_resolve_cname": "false", 2024-09-21T15:32:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_run_sync_thread": "true", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_script_uri": "", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_service_provider_name": "", 2024-09-21T15:32:44.601 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_client_id": "", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_client_secret": "", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_entry": "sts", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_key": "sts", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_auth_url": "", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_custom_header": "", 2024-09-21T15:32:44.602 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_need_stats": "true", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_tenant_name": "", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_url": "", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_lease_period": "120", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T15:32:44.603 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_thread_pool_size": "512", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_comment": "", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_createby": "", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_encoding": "", 2024-09-21T15:32:44.604 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_flag": "false", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_origin": "", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_torrent_tracker": "", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_usage_max_shards": "32", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_use_opa_authz": "false", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T15:32:44.605 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_user_unique_email": "true", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_verify_ssl": "true", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zone": "", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zone_id": "", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T15:32:44.606 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zonegroup": "", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zonegroup_id": "", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_block_size": "4096", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T15:32:44.607 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_partition_filters": "false", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_perf": "false", 2024-09-21T15:32:44.608 INFO:teuthology.orchestra.run.smithi107.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "run_dir": "/var/run/ceph", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_block_create": "true", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_device_size": "53687091200", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T15:32:44.609 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_main_device_type": "SSD", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "seastore_segment_size": "67108864", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "set_keepcaps": "false", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "setgroup": "", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "setuser": "", 2024-09-21T15:32:44.610 INFO:teuthology.orchestra.run.smithi107.stdout: "setuser_match_path": "", 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: "thp": "false", 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: "threadpool_default_timeout": "60", 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout: "throttler_perf_counter": "true" 2024-09-21T15:32:44.611 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:44.613 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T15:32:44.749 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a: {} 2024-09-21T15:32:44.750 INFO:teuthology.orchestra.run.smithi107.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:32:44.762 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b: {} 2024-09-21T15:32:44.762 INFO:teuthology.orchestra.run.smithi107.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:32:44.772 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c: {} 2024-09-21T15:32:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:32:44.786 DEBUG:teuthology.orchestra.run.smithi107:> type ceph-objectstore-tool 2024-09-21T15:32:44.793 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T15:32:44.793 DEBUG:teuthology.orchestra.run.smithi170:> type ceph-objectstore-tool 2024-09-21T15:32:44.798 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T15:32:44.798 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T15:32:44.811 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T15:32:44.811 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T15:32:44.811 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:32:44.812 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:32:44.812 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:32:44.812 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-21T15:32:44.812 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T15:32:44.812 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T15:32:44.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:32:44.813 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T15:32:44.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:32:44.813 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T15:32:44.813 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:32:44.814 DEBUG:teuthology.orchestra.run.smithi107:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-21T15:32:44.837 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:44.836+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:44.855 INFO:teuthology.orchestra.run.smithi107.stdout:mapped writes DISABLED 2024-09-21T15:32:44.855 INFO:teuthology.orchestra.run.smithi107.stdout:mapped reads DISABLED 2024-09-21T15:32:44.855 INFO:teuthology.orchestra.run.smithi107.stdout:Seed set to 761 2024-09-21T15:32:44.915 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:44.911+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:44.973 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:44.974 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:44.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:44.990 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:45.007 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:45.007 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:45.007 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.007 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.016 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:45.012+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:45.026 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:32:45.031 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:45.032 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:45.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:45.052 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:45.071 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:45.072 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:45.092 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:45.092 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:45.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:45.112 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:45.117 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:45.115+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:45.140 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.140 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 18 (0%) 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:avg 2.25 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 1.29904 -> 1.29904 (expected baseline 1.40312) 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.4 with 0 -> 0 pgs (0 -> 0 * mean) 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.1 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:45.142 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:32:45.143 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 0.05 2024-09-21T15:32:45.143 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:32:45.143 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0003 2024-09-21T15:32:45.143 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0003 2024-09-21T15:32:45.143 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:32:45.154 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:32:45.161 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:32:45.161 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-21T15:32:45.252 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:45.252+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:45.312 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.312 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.319 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:45.315+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:45.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:32:45.420 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:45.419+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 18 (0%) 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:avg 2.25 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 1.29904 -> 1.29904 (expected baseline 1.40312) 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.4 with 0 -> 0 pgs (0 -> 0 * mean) 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.1 with 4 -> 4 pgs (1.77778 -> 1.77778 * mean) 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:oload 1000 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 3 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0003 2024-09-21T15:32:45.474 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0031 2024-09-21T15:32:45.475 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:32:45.478 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.478 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:32:45.493 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:32:45.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-21T15:32:45.521 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:45.519+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:45.623 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:45.619+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:45.640 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.641 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:32:45.724 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:45.723+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:45.780 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.780 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:32:45.818 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:32:45.825 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:45.824+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:45.853 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 82 (0%) 2024-09-21T15:32:45.853 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:32:45.853 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-21T15:32:45.853 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 3 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0003 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0003 2024-09-21T15:32:45.854 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:32:45.871 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:32:45.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-21T15:32:45.918 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:45.918 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:45.926 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:45.923+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:45.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:32:46.027 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:46.023+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:32:46.127 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:46.127+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:46.191 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 82 (0%) 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 0.05 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0003 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0003 2024-09-21T15:32:46.192 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:32:46.209 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:32:46.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-21T15:32:46.215 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.215 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.231 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:32:46.231 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:46.227+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:46.332 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:46.331+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:46.379 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.379 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.392 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:32:46.436 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:46.436+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 82 (0%) 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 1 2024-09-21T15:32:46.490 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:32:46.491 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0003 2024-09-21T15:32:46.491 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0003 2024-09-21T15:32:46.491 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:32:46.496 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.496 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:32:46.535 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:46.531+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:46.635 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:46.632+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:46.663 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.663 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.676 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:32:46.736 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:46.736+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:46.818 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.818 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.832 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:32:46.880 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:46.879+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:46.937 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:46.935+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:46.954 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:46.954 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:46.969 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:32:47.038 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:47.036+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:47.129 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:32:47.134 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:47.134 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:47.138 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:47.135+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:47.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:32:47.240 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:47.239+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:47.280 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:47.296 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:47.297 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:47.312 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:47.316 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:47.316 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:47.328 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:47.328 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:47.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:32:47.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:47.348 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:47.371 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:47.371+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:47.387 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:47.387 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:47.405 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:47.405 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:47.442 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:47.439+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:47.503 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:47.503 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:47.503 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:47.503 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:47.503 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:47.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:32:47.560 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:47.559+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:47.645 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:47.643+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:32:47.678 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.891612242000001, 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:32:47.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:32:47.684 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4211.0:1 2.3 e18/17 ondisk, result = 0)", 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.796350+0000", 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.882646221, 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028892799999999998, 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796350+0000", 2024-09-21T15:32:47.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796350+0000", 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796349+0000", 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796365+0000", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6096999999999999e-05 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796389+0000", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3422999999999999e-05 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796402+0000", 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3356999999999999e-05 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796469+0000", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7179000000000002e-05 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796483+0000", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.343e-05 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796639+0000", 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015682599999999999 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.688 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.801305+0000", 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.877692208999999, 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040755700000000001, 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 2 2024-09-21T15:32:47.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801305+0000", 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801305+0000", 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801302+0000", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801320+0000", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.789e-05 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801348+0000", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8106000000000002e-05 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801378+0000", 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9779999999999999e-05 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801496+0000", 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117933 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.801712+0000", 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021603799999999999 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.692 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.802024+0000", 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.876973121000001, 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034489699999999999, 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802024+0000", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802024+0000", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802022+0000", 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:47.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802030+0000", 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4540000000000001e-06 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802050+0000", 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0568999999999999e-05 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802067+0000", 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7286999999999999e-05 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802173+0000", 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105949 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.802368+0000", 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000195055 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:37.790743+0000", 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8882537450000001, 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065532799999999999, 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5 2024-09-21T15:32:47.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790743+0000", 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790743+0000", 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790740+0000", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790758+0000", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8536e-05 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790794+0000", 2024-09-21T15:32:47.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.612e-05 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790813+0000", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8206000000000002e-05 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790921+0000", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108186 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.790931+0000", 2024-09-21T15:32:47.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0413e-05 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.793856+0000", 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002925062 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.794089+0000", 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023258400000000001 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:47.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.794325+0000", 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023603900000000001 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.795659+0000", 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013345290000000001 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797162+0000", 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001502125 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797239+0000", 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7522999999999996e-05 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797296+0000", 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7132999999999997e-05 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.703 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4211.0:5 2.2 e18/17 ondisk, result = 0)", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:37.797024+0000", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8819724600000001, 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000282401, 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797024+0000", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797024+0000", 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797022+0000", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797035+0000", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2934e-05 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797064+0000", 2024-09-21T15:32:47.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9056000000000001e-05 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797078+0000", 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4012999999999999e-05 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797142+0000", 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4146999999999994e-05 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797154+0000", 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1426e-05 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:37.797307+0000", 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015314500000000001 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.8850010360000002, 2024-09-21T15:32:47.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:32:47.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:32:47.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:32:47.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4211.0:7 2.2 e20/17 ondisk, result = 0)", 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.803601+0000", 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.8753957999999997, 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028022499999999999, 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803601+0000", 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803601+0000", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803599+0000", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803610+0000", 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0953e-05 2024-09-21T15:32:47.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803631+0000", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1625999999999999e-05 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803656+0000", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4447000000000002e-05 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803721+0000", 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5083e-05 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803733+0000", 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1885999999999999e-05 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803881+0000", 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014826600000000001 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:47.717 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:47.718 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:47.719 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:32:47.760 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:47.759+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:47.847 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:47.843+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:47.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:32:47.889 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:32:47.890 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:47.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:32:47.948 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:47.947+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:48.049 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:48.047+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:48.070 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:48.070 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:48.071 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:48.071 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:48.071 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:48.084 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:32:48.150 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:48.148+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:48.251 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:48.248+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:48.252 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:48.252 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:48.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:48.252 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [] 2024-09-21T15:32:48.253 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:48.266 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:32:48.352 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:48.352+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:48.383 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:32:48.383 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:48.396 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:32:48.453 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:48.452+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:48.514 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:48.514 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:48.528 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:32:48.571 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:48.571+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:48.650 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:48.650 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:48.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.860072795000001, 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011541064, 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 34 2024-09-21T15:32:48.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787899+0000", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787907+0000", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6539999999999999e-06 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787920+0000", 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2789000000000001e-05 2024-09-21T15:32:48.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787938+0000", 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8277000000000001e-05 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794653+0000", 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067151520000000003 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794782+0000", 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012917899999999999 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794986+0000", 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020385800000000001 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798337+0000", 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033507490000000001 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799304+0000", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00096694399999999996 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799386+0000", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2188999999999995e-05 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799441+0000", 2024-09-21T15:32:48.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5287000000000001e-05 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.860024096, 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011767003, 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.657 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 35 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787948+0000", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788052+0000", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010343700000000001 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788065+0000", 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3436e-05 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788073+0000", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.795624+0000", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075505720000000002 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.795858+0000", 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000234451 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.796219+0000", 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000360305 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799464+0000", 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0032457609999999998 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799544+0000", 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9800000000000002e-05 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799631+0000", 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7201999999999996e-05 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799716+0000", 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4560000000000004e-05 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.859888953999999, 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011912222, 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 36 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.663 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788083+0000", 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788104+0000", 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1134000000000001e-05 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788115+0000", 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0135999999999999e-05 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788122+0000", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5599999999999996e-06 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797472+0000", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093495780000000007 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797640+0000", 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000168307 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797814+0000", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174222 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799484+0000", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001670019 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799875+0000", 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039030400000000001 2024-09-21T15:32:48.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799941+0000", 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6632999999999997e-05 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799996+0000", 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5036e-05 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.859839591, 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012487280999999999, 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:48.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788139+0000", 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788148+0000", 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2860000000000005e-06 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788155+0000", 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8469999999999998e-06 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798408+0000", 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010253010999999999 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798541+0000", 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013288899999999999 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798735+0000", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000194351 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800035+0000", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001299854 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800395+0000", 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036033000000000001 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800524+0000", 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012841600000000001 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800621+0000", 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6842999999999994e-05 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.763540077000002, 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011676723, 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 87 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884432+0000", 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884525+0000", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3066000000000006e-05 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884539+0000", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3253e-05 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884546+0000", 2024-09-21T15:32:48.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8869999999999992e-06 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.891565+0000", 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0070186390000000001 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.891792+0000", 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022678800000000001 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.892195+0000", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000403407 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894459+0000", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022640289999999999 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895975+0000", 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001515366 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896052+0000", 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7169000000000006e-05 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896110+0000", 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7756000000000001e-05 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.763415765000001, 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012694309000000001, 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 88 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884556+0000", 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884563+0000", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5969999999999997e-06 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884572+0000", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2599999999999994e-06 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884582+0000", 2024-09-21T15:32:48.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4299999999999995e-06 2024-09-21T15:32:48.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894521+0000", 2024-09-21T15:32:48.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099389100000000005 2024-09-21T15:32:48.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894744+0000", 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337400000000001 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895106+0000", 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000361861 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896480+0000", 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013741000000000001 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897024+0000", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054449900000000005 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897153+0000", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000129016 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897252+0000", 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8091999999999995e-05 2024-09-21T15:32:48.687 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.584554996000001, 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011288622999999999, 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.688 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 110 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063417+0000", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063430+0000", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.327e-05 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063449+0000", 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9556999999999999e-05 2024-09-21T15:32:48.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063464+0000", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.412e-05 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.069816+0000", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063522939999999996 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.069925+0000", 2024-09-21T15:32:48.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010952599999999999 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.070105+0000", 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017942500000000001 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072118+0000", 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020134939999999998 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074588+0000", 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024696879999999998 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074656+0000", 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8442999999999995e-05 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074707+0000", 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0219000000000001e-05 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.693 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.584473673000002, 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011533918000000001, 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 111 2024-09-21T15:32:48.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.695 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063498+0000", 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063510+0000", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.239e-05 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063530+0000", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9236999999999999e-05 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063544+0000", 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4365999999999999e-05 2024-09-21T15:32:48.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071126+0000", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075817189999999998 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071270+0000", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014477500000000001 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071468+0000", 2024-09-21T15:32:48.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019767799999999999 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074797+0000", 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033287849999999999 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074910+0000", 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011262199999999999 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074971+0000", 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1456000000000002e-05 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075033+0000", 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2180000000000004e-05 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:48.700 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.584415437000001, 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011602988, 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 112 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063557+0000", 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063658+0000", 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010134899999999999 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063672+0000", 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3477e-05 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063680+0000", 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3760000000000008e-06 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072503+0000", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0088231690000000005 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072642+0000", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000138512 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072835+0000", 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019289200000000001 2024-09-21T15:32:48.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074891+0000", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020561580000000002 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075060+0000", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016887200000000001 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075120+0000", 2024-09-21T15:32:48.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0078999999999999e-05 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075161+0000", 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0776999999999999e-05 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:48.706 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.584282443999999, 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013906523, 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063690+0000", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063696+0000", 2024-09-21T15:32:48.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8830000000000003e-06 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063705+0000", 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063712+0000", 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1999999999999997e-06 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073726+0000", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010013986000000001 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073852+0000", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125699 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074033+0000", 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000181502 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077367+0000", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033334409999999999 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077478+0000", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110789 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077545+0000", 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7490000000000006e-05 2024-09-21T15:32:48.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077597+0000", 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.584253051000001, 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014011229, 2024-09-21T15:32:48.712 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063719+0000", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063725+0000", 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6069999999999998e-06 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063734+0000", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063741+0000", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9129999999999997e-06 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075299+0000", 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011557827999999999 2024-09-21T15:32:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075415+0000", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116066 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075607+0000", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019232099999999999 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077458+0000", 2024-09-21T15:32:48.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001851339 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077615+0000", 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015717899999999999 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077677+0000", 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1467e-05 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077731+0000", 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4196000000000001e-05 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.583705951999999, 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015388028, 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 115 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:48.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064266+0000", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064384+0000", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117656 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064400+0000", 2024-09-21T15:32:48.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5760000000000002e-05 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064409+0000", 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4429999999999992e-06 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076158+0000", 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011749226 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076294+0000", 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136122 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076470+0000", 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017574600000000001 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079442+0000", 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002971577 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079509+0000", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7346000000000002e-05 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079582+0000", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3412999999999997e-05 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079655+0000", 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2649000000000002e-05 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.723 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.583552733000001, 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015786732000000001, 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 116 2024-09-21T15:32:48.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064419+0000", 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064428+0000", 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.986e-06 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064438+0000", 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8039999999999996e-06 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064445+0000", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9789999999999996e-06 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077804+0000", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013359207999999999 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077921+0000", 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117039 2024-09-21T15:32:48.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078131+0000", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000209395 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079687+0000", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001555758 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080102+0000", 2024-09-21T15:32:48.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000415547 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080163+0000", 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0705999999999998e-05 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080207+0000", 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4202999999999997e-05 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.583520100000001, 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:32:48.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116407759000001, 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013241467, 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 136 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531570+0000", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0730000000000004e-06 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531579+0000", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1079999999999999e-06 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531585+0000", 2024-09-21T15:32:48.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6730000000000003e-06 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536711+0000", 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0051255220000000004 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536808+0000", 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6739999999999999e-05 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536919+0000", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110918 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539177+0000", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002258262 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544733+0000", 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055560599999999998 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544767+0000", 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3918000000000003e-05 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544807+0000", 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9753999999999999e-05 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116381452999999, 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013385154, 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 137 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531591+0000", 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531654+0000", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2352000000000002e-05 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531660+0000", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1569999999999997e-06 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531663+0000", 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6480000000000001e-06 2024-09-21T15:32:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537277+0000", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056140290000000004 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537387+0000", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109131 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537569+0000", 2024-09-21T15:32:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182051 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540762+0000", 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031933679999999998 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544872+0000", 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041098319999999999 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544927+0000", 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5470999999999998e-05 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544977+0000", 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9388999999999998e-05 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116304572000001, 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013408308000000001, 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 138 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531671+0000", 2024-09-21T15:32:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7659999999999999e-06 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531674+0000", 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5499999999999999e-06 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531677+0000", 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0290000000000001e-06 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538356+0000", 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066784920000000003 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538462+0000", 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106572 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538622+0000", 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015995899999999999 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541174+0000", 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025514589999999998 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545024+0000", 2024-09-21T15:32:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0038503629999999999 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545055+0000", 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0278999999999999e-05 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545077+0000", 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2177000000000001e-05 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116292668, 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013559554, 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 139 2024-09-21T15:32:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967296 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531682+0000", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1280000000000002e-06 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531687+0000", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4970000000000003e-06 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531694+0000", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4560000000000003e-06 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539257+0000", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007562966 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539341+0000", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4319000000000004e-05 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539437+0000", 2024-09-21T15:32:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5561000000000002e-05 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541183+0000", 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017459629999999999 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545135+0000", 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0039523300000000004 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545194+0000", 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8485e-05 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545240+0000", 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6060999999999998e-05 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116264959999999, 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013608026, 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 140 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531757+0000", 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9517e-05 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531765+0000", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6960000000000005e-06 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531769+0000", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6380000000000002e-06 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539783+0000", 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080140839999999994 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539891+0000", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010794900000000001 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540060+0000", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016991099999999999 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541649+0000", 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015888370000000001 2024-09-21T15:32:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545266+0000", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036169309999999999 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545294+0000", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8170999999999998e-05 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545316+0000", 2024-09-21T15:32:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1642e-05 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.116199643000002, 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014580891, 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 141 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531776+0000", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531779+0000", 2024-09-21T15:32:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6179999999999999e-06 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531782+0000", 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541202+0000", 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094203670000000007 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541258+0000", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6283999999999998e-05 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541355+0000", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6038000000000004e-05 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.542023+0000", 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000668656 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546250+0000", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042266659999999996 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546307+0000", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6805999999999998e-05 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546354+0000", 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.757e-05 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:48.771 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:48.772 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:32:48.773 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:48.771+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:48.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:32:48.873 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:48.872+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:48.937 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:48.938 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:48.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:32:48.974 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:48.971+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:49.075 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:49.072+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:49.117 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:32:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:32:49.118 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:49.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:32:49.216 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:49.216+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:49.277 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:49.275+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.732314+0000", 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.565896242000001, 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022627229999999999, 2024-09-21T15:32:49.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732314+0000", 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732314+0000", 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732313+0000", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732321+0000", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8599999999999993e-06 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732347+0000", 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6582999999999999e-05 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732363+0000", 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5936000000000001e-05 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732498+0000", 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013448299999999999 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.732534+0000", 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6198999999999997e-05 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.734490+0000", 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019558380000000001 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.734576+0000", 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6349999999999998e-05 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'17)", 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.793312+0000", 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.504897616000001, 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022658750000000001, 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793312+0000", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793312+0000", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793310+0000", 2024-09-21T15:32:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793324+0000", 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3465999999999999e-05 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793353+0000", 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793366+0000", 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.236e-05 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793430+0000", 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4078999999999994e-05 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.793464+0000", 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.43e-05 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.795496+0000", 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020324039999999998 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.795578+0000", 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1768999999999996e-05 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'17)", 2024-09-21T15:32:49.310 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.794579+0000", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.503631008999999, 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022400689999999999, 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794579+0000", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794579+0000", 2024-09-21T15:32:49.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794577+0000", 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794667+0000", 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0173000000000006e-05 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794699+0000", 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1986000000000002e-05 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794718+0000", 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9199999999999999e-05 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794802+0000", 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3345999999999998e-05 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.794836+0000", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4353000000000001e-05 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.796728+0000", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018915290000000001 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.796819+0000", 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1318999999999998e-05 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:68 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'42, mlcod=15'39)", 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.857120+0000", 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.441090201000002, 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002437214, 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857120+0000", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857120+0000", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857118+0000", 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857128+0000", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9399999999999997e-06 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857149+0000", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1152999999999999e-05 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857159+0000", 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7399999999999999e-06 2024-09-21T15:32:49.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857276+0000", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116406 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.857337+0000", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1218999999999998e-05 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.859487+0000", 2024-09-21T15:32:49.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021505640000000002 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.859557+0000", 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9546000000000001e-05 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:69 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.858325+0000", 2024-09-21T15:32:49.319 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.439884624000001, 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027272849999999999, 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858325+0000", 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858325+0000", 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858323+0000", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858380+0000", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6433e-05 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858410+0000", 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9804000000000001e-05 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858422+0000", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2463000000000001e-05 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858525+0000", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102469 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.858595+0000", 2024-09-21T15:32:49.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0369000000000003e-05 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.860963+0000", 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002367905 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.861052+0000", 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9676e-05 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:84 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'54, mlcod=15'50)", 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:30.885750+0000", 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.412459589000001, 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022853420000000001, 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885750+0000", 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885750+0000", 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885748+0000", 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885802+0000", 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3798999999999998e-05 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885827+0000", 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4709999999999999e-05 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885839+0000", 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2413e-05 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885920+0000", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0568999999999994e-05 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.885959+0000", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9079999999999999e-05 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.887917+0000", 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001957702 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:30.888036+0000", 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011890199999999999 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:130 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'88, mlcod=15'85)", 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.530313+0000", 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.767896768, 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029098840000000002, 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530313+0000", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530313+0000", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530311+0000", 2024-09-21T15:32:49.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530329+0000", 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7723999999999998e-05 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530359+0000", 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9606000000000001e-05 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530371+0000", 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2563e-05 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530447+0000", 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5523000000000002e-05 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.530478+0000", 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.116e-05 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533166+0000", 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026883789999999999 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533223+0000", 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6783000000000002e-05 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.332 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:131 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'89, mlcod=15'85)", 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.531023+0000", 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.76718692, 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045159520000000002, 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531023+0000", 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531023+0000", 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531021+0000", 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531161+0000", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014033200000000001 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531191+0000", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9796000000000001e-05 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531214+0000", 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2935999999999999e-05 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531319+0000", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105133 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531378+0000", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.889e-05 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535449+0000", 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040711949999999997 2024-09-21T15:32:49.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535539+0000", 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9612999999999997e-05 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.766277317, 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055491940000000003, 2024-09-21T15:32:49.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:49.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531931+0000", 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531944+0000", 2024-09-21T15:32:49.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3793e-05 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531977+0000", 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2493000000000002e-05 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531990+0000", 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2799000000000001e-05 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532542+0000", 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055232999999999997 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532585+0000", 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3040000000000001e-05 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537415+0000", 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048296249999999997 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537482+0000", 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6987000000000001e-05 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:133 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'91, mlcod=15'86)", 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.532733+0000", 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.765476360000001, 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048184059999999999, 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532733+0000", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532733+0000", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532732+0000", 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532745+0000", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2969999999999999e-05 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532774+0000", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9643000000000001e-05 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532788+0000", 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.344e-05 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533313+0000", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052543200000000005 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533365+0000", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.168e-05 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537489+0000", 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041236750000000003 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537552+0000", 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3266e-05 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.764861061000001, 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060115560000000004, 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533347+0000", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533427+0000", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9798999999999999e-05 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533447+0000", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0207000000000002e-05 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533461+0000", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3413e-05 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534231+0000", 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077009699999999997 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534286+0000", 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5556999999999997e-05 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539301+0000", 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050144669999999999 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539360+0000", 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9472999999999998e-05 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:135 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'93, mlcod=15'86)", 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.534334+0000", 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.763875765999998, 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051117309999999996, 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.351 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534334+0000", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534334+0000", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534332+0000", 2024-09-21T15:32:49.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534346+0000", 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3997e-05 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534371+0000", 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5089999999999999e-05 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534386+0000", 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4813e-05 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535649+0000", 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012628330000000001 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535722+0000", 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2810000000000003e-05 2024-09-21T15:32:49.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539367+0000", 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003645045 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539446+0000", 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8953000000000001e-05 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.762979900000001, 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.359 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.762407824, 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:32:49.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:32:49.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'87)", 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.761269782999999, 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058430519999999996, 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.365 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536938+0000", 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536962+0000", 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.353e-05 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537014+0000", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2676000000000002e-05 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537041+0000", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538691+0000", 2024-09-21T15:32:49.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016501980000000001 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538726+0000", 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5573e-05 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542722+0000", 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039951259999999999 2024-09-21T15:32:49.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542783+0000", 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1465999999999997e-05 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:139 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'97, mlcod=15'88)", 2024-09-21T15:32:49.370 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.537696+0000", 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.760514018999999, 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051561510000000003, 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537696+0000", 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537696+0000", 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537695+0000", 2024-09-21T15:32:49.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537706+0000", 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1327e-05 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537723+0000", 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6762999999999999e-05 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537734+0000", 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.146e-05 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539566+0000", 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018319790000000001 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539620+0000", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4106e-05 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542786+0000", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003165869 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542852+0000", 2024-09-21T15:32:49.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5950000000000004e-05 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:140 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'98, mlcod=15'88)", 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.538384+0000", 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.759825704000001, 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045806149999999997, 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538384+0000", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538384+0000", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538382+0000", 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538440+0000", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7485999999999998e-05 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538465+0000", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5259999999999999e-05 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538478+0000", 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2883e-05 2024-09-21T15:32:49.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.540324+0000", 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001846456 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.540361+0000", 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6989000000000001e-05 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542896+0000", 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002534344 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542965+0000", 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8960000000000004e-05 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:141 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'99, mlcod=15'90)", 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.539580+0000", 2024-09-21T15:32:49.380 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.758629630000001, 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216909999999998, 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539580+0000", 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539580+0000", 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539579+0000", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539589+0000", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0274e-05 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539609+0000", 2024-09-21T15:32:49.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9695999999999999e-05 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539622+0000", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3617e-05 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542082+0000", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024590509999999999 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542122+0000", 2024-09-21T15:32:49.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0645999999999997e-05 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.544119+0000", 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019965489999999998 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.544202+0000", 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3145000000000004e-05 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:142 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'100, mlcod=15'98)", 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.545057+0000", 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.753152377999999, 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002594846, 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545057+0000", 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545057+0000", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545056+0000", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545123+0000", 2024-09-21T15:32:49.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7232999999999998e-05 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545152+0000", 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9465999999999999e-05 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545165+0000", 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2774e-05 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545282+0000", 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117192 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.545337+0000", 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4583000000000002e-05 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.547582+0000", 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022450619999999999 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.547652+0000", 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0432999999999995e-05 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.745369+0000", 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.55284075399999999, 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025405279999999998, 2024-09-21T15:32:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745369+0000", 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745369+0000", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745366+0000", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745381+0000", 2024-09-21T15:32:49.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4567e-05 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745409+0000", 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.815e-05 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745424+0000", 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4916000000000001e-05 2024-09-21T15:32:49.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745555+0000", 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000130617 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.745770+0000", 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021581700000000001 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:49.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.746047+0000", 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027665900000000001 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.747115+0000", 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001067798 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:49.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.747822+0000", 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00070698799999999997 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.747870+0000", 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.747910+0000", 2024-09-21T15:32:49.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9412999999999997e-05 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:49.396 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:49.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:32:49.398 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:49.395+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:49.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:32:49.498 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:49.495+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:49.561 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:49.561 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:49.561 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:49.562 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:49.575 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:32:49.581 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:49.581 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:49.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:49.599 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:49.599 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:49.599+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:49.614 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:49.615 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:49.632 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:49.632 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:49.652 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:49.652 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:49.672 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:49.686 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:49.686 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:49.692 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:49.692 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:49.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:32:49.701 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:49.699+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:49.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:49.713 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:49.801 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:49.799+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:49.825 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:49.825 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:49.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:32:49.902 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:49.899+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:49.957 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:49.957 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:49.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:32:50.003 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:50.003+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:50.102 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.102 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.104 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:50.103+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:50.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:32:50.204 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:50.203+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:50.225 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.226 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.239 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:32:50.305 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:50.303+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:50.348 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.348 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:32:50.406 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:50.404+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:50.470 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.470 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:32:50.528 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:50.528+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:50.608 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:50.607+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.641 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.655 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:32:50.702 INFO:teuthology.orchestra.run.smithi107.stdout:1 write 0x8a914cb thru 0x8aa132e (0xfe64 bytes) 2024-09-21T15:32:50.707 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:50.708 INFO:teuthology.orchestra.run.smithi107.stdout:3 write 0x64f6817 thru 0x64fa389 (0x3b73 bytes) 2024-09-21T15:32:50.708 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:50.707+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:50.726 INFO:teuthology.orchestra.run.smithi107.stdout:4 write 0xab70040 thru 0xab78ffc (0x8fbd bytes) 2024-09-21T15:32:50.729 INFO:teuthology.orchestra.run.smithi107.stdout:5 writesame 0xa3b947a thru 0xa3c6824 data_size 0x1(0xd3ab bytes) 2024-09-21T15:32:50.767 INFO:teuthology.orchestra.run.smithi107.stdout:6 read 0x64b3f61 thru 0x64c0010 (0xc0b0 bytes) 2024-09-21T15:32:50.768 INFO:teuthology.orchestra.run.smithi107.stdout:8 write 0x2ae5370 thru 0x2ae5a5e (0x6ef bytes) 2024-09-21T15:32:50.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:32:50.775 INFO:teuthology.orchestra.run.smithi107.stdout:9 read 0x4313ebd thru 0x431719f (0x32e3 bytes) 2024-09-21T15:32:50.776 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0x60510fc 2024-09-21T15:32:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:11 trunc from 0xab78ffd to 0x60510fc 2024-09-21T15:32:50.792 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.793 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.802 INFO:teuthology.orchestra.run.smithi107.stdout:12 clone 1 order 21 su 8192 sc 2 2024-09-21T15:32:50.809 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:32:50.809 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:50.808+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:50.910 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:50.907+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:50.976 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:50.976 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:50.991 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:32:51.027 INFO:teuthology.orchestra.run.smithi107.stdout:13 writesame 0xdd786c8 thru 0xdd846f3 data_size 0x62(0xc02c bytes) 2024-09-21T15:32:51.027 INFO:teuthology.orchestra.run.smithi107.stdout:14 write 0x4e07002 thru 0x4e0d7c3 (0x67c2 bytes) 2024-09-21T15:32:51.034 INFO:teuthology.orchestra.run.smithi107.stdout:15 read 0xd64efa4 thru 0xd657142 (0x819f bytes) 2024-09-21T15:32:51.034 INFO:teuthology.orchestra.run.smithi107.stdout:17 read 0x3548157 thru 0x354a945 (0x27ef bytes) 2024-09-21T15:32:51.034 INFO:teuthology.orchestra.run.smithi107.stdout:19 punch from 0x4d304b8 to 0x4d35dca, (0x5912 bytes) 2024-09-21T15:32:51.035 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:51.035+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:51.051 INFO:teuthology.orchestra.run.smithi107.stdout:21 trunc from 0xdd846f4 to 0x15dc2c1 2024-09-21T15:32:51.085 INFO:teuthology.orchestra.run.smithi107.stdout:24 writesame 0x32f0cae thru 0x32f6ef8 data_size 0x1(0x624b bytes) 2024-09-21T15:32:51.085 INFO:teuthology.orchestra.run.smithi107.stdout:25 read 0x1d25c0 thru 0x1d43ec (0x1e2d bytes) 2024-09-21T15:32:51.109 INFO:teuthology.orchestra.run.smithi107.stdout:26 write 0xbe35dc9 thru 0xbe39b39 (0x3d71 bytes) 2024-09-21T15:32:51.112 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:51.112+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:51.134 INFO:teuthology.orchestra.run.smithi107.stdout:29 write 0xe54f78e thru 0xe550b5e (0x13d1 bytes) 2024-09-21T15:32:51.139 INFO:teuthology.orchestra.run.smithi107.stdout:30 writesame 0x7bcfdee thru 0x7bd1043 data_size 0x1(0x1256 bytes) 2024-09-21T15:32:51.149 INFO:teuthology.orchestra.run.smithi107.stdout:32 write 0x8f2d473 thru 0x8f388f8 (0xb486 bytes) 2024-09-21T15:32:51.157 INFO:teuthology.orchestra.run.smithi107.stdout:35 read 0x74c35a6 thru 0x74cf890 (0xc2eb bytes) 2024-09-21T15:32:51.157 INFO:teuthology.orchestra.run.smithi107.stdout:36 punch from 0xd9a746a to 0xd9ad102, (0x5c98 bytes) 2024-09-21T15:32:51.157 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.157 INFO:teuthology.orchestra.run.smithi107.stdout:39 read 0x73e2226 thru 0x73e6425 (0x4200 bytes) 2024-09-21T15:32:51.157 INFO:teuthology.orchestra.run.smithi107.stdout:40 read 0x7049458 thru 0x70517a8 (0x8351 bytes) 2024-09-21T15:32:51.158 INFO:teuthology.orchestra.run.smithi107.stdout:43 writesame 0xe112de1 thru 0xe113d40 data_size 0x18(0xf60 bytes) 2024-09-21T15:32:51.163 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.164 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.164 INFO:teuthology.orchestra.run.smithi107.stdout:49 write 0x82c2d97 thru 0x82cf748 (0xc9b2 bytes) 2024-09-21T15:32:51.167 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:51.167 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:51.172 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.173 INFO:teuthology.orchestra.run.smithi107.stdout:51 writesame 0x3dbed86 thru 0x3dc9f6e data_size 0x1(0xb1e9 bytes) 2024-09-21T15:32:51.183 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:32:51.205 INFO:teuthology.orchestra.run.smithi107.stdout:53 write 0xbe5b9b5 thru 0xbe6861d (0xcc69 bytes) 2024-09-21T15:32:51.208 INFO:teuthology.orchestra.run.smithi107.stdout:55 punch from 0x396fc59 to 0x397114a, (0x14f1 bytes) 2024-09-21T15:32:51.208 INFO:teuthology.orchestra.run.smithi107.stdout:56 read 0xa077626 thru 0xa08696d (0xf348 bytes) 2024-09-21T15:32:51.208 INFO:teuthology.orchestra.run.smithi107.stdout:57 write 0xe23a4d3 thru 0xe23e6b7 (0x41e5 bytes) 2024-09-21T15:32:51.213 INFO:teuthology.orchestra.run.smithi107.stdout:58 write 0x6177f28 thru 0x617ec06 (0x6cdf bytes) 2024-09-21T15:32:51.221 INFO:teuthology.orchestra.run.smithi107.stdout:59 writesame 0xd0b2b46 thru 0xd0b83e1 data_size 0x4(0x589c bytes) 2024-09-21T15:32:51.227 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:51.227+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:51.231 INFO:teuthology.orchestra.run.smithi107.stdout:60 write 0x395dca6 thru 0x3962ea4 (0x51ff bytes) 2024-09-21T15:32:51.241 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.241 INFO:teuthology.orchestra.run.smithi107.stdout:62 write 0xc9b7465 thru 0xc9bffdc (0x8b78 bytes) 2024-09-21T15:32:51.249 INFO:teuthology.orchestra.run.smithi107.stdout:63 writesame 0x76ec603 thru 0x76f6d79 data_size 0x1(0xa777 bytes) 2024-09-21T15:32:51.285 INFO:teuthology.orchestra.run.smithi107.stdout:64 punch from 0x7ec0365 to 0x7ec4254, (0x3eef bytes) 2024-09-21T15:32:51.285 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.285 INFO:teuthology.orchestra.run.smithi107.stdout:67 write 0x985a2be thru 0x98675b1 (0xd2f4 bytes) 2024-09-21T15:32:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:69 writesame 0xea6c774 thru 0xea778eb data_size 0x24(0xb178 bytes) 2024-09-21T15:32:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:70 punch from 0x94fe8de to 0x9500ec4, (0x25e6 bytes) 2024-09-21T15:32:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:71 punch from 0x3f89a4d to 0x3f8c874, (0x2e27 bytes) 2024-09-21T15:32:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:72 read 0x240e494 thru 0x2417de0 (0x994d bytes) 2024-09-21T15:32:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0x9a07478 2024-09-21T15:32:51.309 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:51.309 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:51.313 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:51.311+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:51.320 INFO:teuthology.orchestra.run.smithi107.stdout:73 trunc from 0xea778ec to 0x9a07478 2024-09-21T15:32:51.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:32:51.345 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.345 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.346 INFO:teuthology.orchestra.run.smithi107.stdout:77 read 0xabe6d2 thru 0xacd556 (0xee85 bytes) 2024-09-21T15:32:51.346 INFO:teuthology.orchestra.run.smithi107.stdout:78 write 0x346c496 thru 0x3476f9d (0xab08 bytes) 2024-09-21T15:32:51.353 INFO:teuthology.orchestra.run.smithi107.stdout:80 write 0x17b9de3 thru 0x17bdc97 (0x3eb5 bytes) 2024-09-21T15:32:51.356 INFO:teuthology.orchestra.run.smithi107.stdout:81 read 0xe6ffb0 thru 0xe7a474 (0xa4c5 bytes) 2024-09-21T15:32:51.356 INFO:teuthology.orchestra.run.smithi107.stdout:82 read 0x6aa0550 thru 0x6aa2815 (0x22c6 bytes) 2024-09-21T15:32:51.357 INFO:teuthology.orchestra.run.smithi107.stdout:83 writesame 0x41c00d8 thru 0x41ca108 data_size 0x1(0xa031 bytes) 2024-09-21T15:32:51.409 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:51.409 INFO:teuthology.orchestra.run.smithi107.stdout:87 write 0x748fdf5 thru 0x749d284 (0xd490 bytes) 2024-09-21T15:32:51.412 INFO:teuthology.orchestra.run.smithi107.stdout:88 punch from 0x187c23f to 0x1881f8d, (0x5d4e bytes) 2024-09-21T15:32:51.412 INFO:teuthology.orchestra.run.smithi107.stdout:92 clone 2 order 20 su 262144 sc 8 2024-09-21T15:32:51.413 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:51.411+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:51.490 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:51.490 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:51.504 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:32:51.510 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:32:51.510 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:32:51.510 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:32:51.510 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:32:51.556 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:51.556+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:51.615 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:51.611+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:51.670 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:51.670 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:51.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:32:51.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:32:51.732 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:51.731+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 25, 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:32:50.827730+0000", 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:32:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:32:19.548914+0000", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5806080, 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "nodeep-scrub", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "noscrub", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:32:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "jewel", 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:32:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:32:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 8, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 8, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.25, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.9900000095367432, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.9900000095367432, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "25", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 4, 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 25, 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.5, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.5, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.5, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.5, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:51.836 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:32:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:32:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:32:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:32:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 17, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:32:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:32:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:32:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:32:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:32:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:32:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:32:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:32:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:32:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:32:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [ 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 2, 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 1 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:51.890 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:51.891 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:51.816+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:51.892 INFO:tasks.thrashosds.thrasher:j is {'epoch': 25, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:32:50.827730+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:32:19.548914+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 25, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:32:51.892 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T15:32:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:32:51.893 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:51.895 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:51.895 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:51.895 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:32:51.907 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:51.907 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:51.916 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone1 2024-09-21T15:32:51.925 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:51.925 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:51.942 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:51.942 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:51.943 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:51.943+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:51.960 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:51.960 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:51.979 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:51.980 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:51.999 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:51.999 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:52.018 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:52.015+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:52.033 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:52.033 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:52.033 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:52.033 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:52.033 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:52.048 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:32:52.070 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T15:32:52.118 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:52.116+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:52.219 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:52.219+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4697031640000002, 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072729969999999998, 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.222 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751175+0000", 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751188+0000", 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3774000000000001e-05 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751208+0000", 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0173000000000002e-05 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751232+0000", 2024-09-21T15:32:52.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3059999999999999e-05 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751344+0000", 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011212199999999999 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751354+0000", 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9199999999999999e-06 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755656+0000", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043023430000000001 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755870+0000", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000213809 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.756264+0000", 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039450599999999998 2024-09-21T15:32:52.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.757715+0000", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001451232 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758252+0000", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053677 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758350+0000", 2024-09-21T15:32:52.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7275000000000006e-05 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758450+0000", 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100603 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.765885+0000", 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4549948869999998, 2024-09-21T15:32:52.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00068342800000000003, 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765885+0000", 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765885+0000", 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765883+0000", 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765896+0000", 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2649e-05 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765921+0000", 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5154e-05 2024-09-21T15:32:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.765942+0000", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1180000000000001e-05 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766031+0000", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9061999999999995e-05 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766201+0000", 2024-09-21T15:32:52.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000169199 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766569+0000", 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000368134 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.234 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:17 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.766781+0000", 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4540989010000001, 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00048014299999999999, 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 17 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.235 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766781+0000", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766781+0000", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766779+0000", 2024-09-21T15:32:52.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766789+0000", 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0016e-05 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766812+0000", 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2467e-05 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766831+0000", 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9199999999999999e-05 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.766921+0000", 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9602e-05 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767063+0000", 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014191200000000001 2024-09-21T15:32:52.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767262+0000", 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019890899999999999 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:18 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.767439+0000", 2024-09-21T15:32:52.239 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.453441363, 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00048879000000000001, 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 18 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767439+0000", 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767439+0000", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767437+0000", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767447+0000", 2024-09-21T15:32:52.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.011e-05 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767467+0000", 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9806e-05 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767486+0000", 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8340000000000001e-05 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767559+0000", 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3252999999999998e-05 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767686+0000", 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000127429 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.767928+0000", 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024149499999999999 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:20 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.769021+0000", 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4518597579999999, 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00073917999999999998, 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 20 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.244 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769021+0000", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769021+0000", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769019+0000", 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769027+0000", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0139999999999998e-06 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769047+0000", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9219999999999999e-05 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769064+0000", 2024-09-21T15:32:52.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.791e-05 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769122+0000", 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7889000000000001e-05 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769250+0000", 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000127892 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769760+0000", 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00050944900000000003 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:21 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.769952+0000", 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4509282150000002, 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00065360799999999997, 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 21 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769952+0000", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769952+0000", 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769951+0000", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769960+0000", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6039999999999999e-06 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.769979+0000", 2024-09-21T15:32:52.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9103e-05 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.770006+0000", 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789000000000001e-05 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.770074+0000", 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7543e-05 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.770216+0000", 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000141989 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.770606+0000", 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038998400000000001 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:24 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.772428+0000", 2024-09-21T15:32:52.252 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.448452198, 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00049146300000000003, 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 24 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772428+0000", 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772428+0000", 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772427+0000", 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772437+0000", 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0027000000000001e-05 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772457+0000", 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0599999999999999e-05 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772475+0000", 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7413000000000001e-05 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772578+0000", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103039 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772713+0000", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013531900000000001 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.772920+0000", 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000206648 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.256 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:25 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [watch watch cookie 140139749721184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4476664000000001, 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027829349999999998, 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 25 2024-09-21T15:32:52.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773212+0000", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773236+0000", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3326999999999999e-05 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773255+0000", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9862999999999999e-05 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773295+0000", 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773378+0000", 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2818999999999994e-05 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773511+0000", 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132669 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773743+0000", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023192500000000001 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.774930+0000", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011868510000000001 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775664+0000", 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00073456699999999999 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775799+0000", 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013473900000000001 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775997+0000", 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019791899999999999 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.262 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:25 3.a e24/21 ondisk, result = 0)", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.775471+0000", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.445408987, 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054155200000000005, 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775471+0000", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775471+0000", 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775470+0000", 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775481+0000", 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1206000000000001e-05 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775506+0000", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4672999999999999e-05 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775518+0000", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2819999999999999e-05 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775622+0000", 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103923 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775647+0000", 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5086e-05 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.776013+0000", 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000365514 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:495 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.266 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.061358+0000", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15952197900000001, 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00075053800000000003, 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 495 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061358+0000", 2024-09-21T15:32:52.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061358+0000", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061354+0000", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061376+0000", 2024-09-21T15:32:52.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1520000000000001e-05 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061410+0000", 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4332999999999998e-05 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061426+0000", 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5860000000000001e-05 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061552+0000", 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012580499999999999 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.061712+0000", 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016034299999999999 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062109+0000", 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000396497 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:496 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.062279+0000", 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15860099599999999, 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045242299999999997, 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 496 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062279+0000", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062279+0000", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062278+0000", 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062287+0000", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9530000000000005e-06 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062307+0000", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9933e-05 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062319+0000", 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2743e-05 2024-09-21T15:32:52.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062417+0000", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7260000000000001e-05 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062541+0000", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124499 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062732+0000", 2024-09-21T15:32:52.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000190588 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:497 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.062898+0000", 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.157981817, 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00050872899999999997, 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.275 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 497 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062898+0000", 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062898+0000", 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062897+0000", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062905+0000", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1359999999999997e-06 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062923+0000", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7439999999999999e-05 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.062935+0000", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2563e-05 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063030+0000", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4506000000000006e-05 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063154+0000", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012400900000000001 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063407+0000", 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00025357500000000002 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:498 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.063703+0000", 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15717747400000001, 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054933900000000003, 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.279 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 498 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063703+0000", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063703+0000", 2024-09-21T15:32:52.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063702+0000", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063711+0000", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7629999999999995e-06 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063728+0000", 2024-09-21T15:32:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6413e-05 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063739+0000", 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0832999999999999e-05 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.063897+0000", 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015872299999999999 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064039+0000", 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014167499999999999 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064252+0000", 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021306499999999999 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:499 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.064670+0000", 2024-09-21T15:32:52.283 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15620979500000001, 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00086947299999999995, 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 499 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064670+0000", 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064670+0000", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064669+0000", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064677+0000", 2024-09-21T15:32:52.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.52e-06 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064693+0000", 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6229999999999999e-05 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064705+0000", 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2254e-05 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064807+0000", 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000102365 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.064951+0000", 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000143423 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065540+0000", 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00058919099999999998 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:500 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.065676+0000", 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15520410300000001, 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00076574399999999995, 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 500 2024-09-21T15:32:52.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065676+0000", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065676+0000", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065675+0000", 2024-09-21T15:32:52.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065682+0000", 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6660000000000008e-06 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065698+0000", 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.59e-05 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065713+0000", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.429e-05 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065813+0000", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010048300000000001 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.065941+0000", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128489 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066442+0000", 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00050043199999999998 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:501 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.066651+0000", 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.154228903, 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00058137200000000003, 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 501 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066651+0000", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066651+0000", 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066650+0000", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066658+0000", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7370000000000006e-06 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066675+0000", 2024-09-21T15:32:52.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7116e-05 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066703+0000", 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8453e-05 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066858+0000", 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000155229 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.066997+0000", 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013825900000000001 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.067233+0000", 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236145 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:503 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.068182+0000", 2024-09-21T15:32:52.296 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15269812799999999, 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053491599999999997, 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 503 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068182+0000", 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068182+0000", 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068181+0000", 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068188+0000", 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068205+0000", 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7113e-05 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068214+0000", 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7339999999999997e-06 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068367+0000", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015285200000000001 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068506+0000", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013922500000000001 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.068717+0000", 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000211339 2024-09-21T15:32:52.300 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:504 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.069019+0000", 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.151861044, 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000812757, 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.301 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 504 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069019+0000", 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069019+0000", 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069018+0000", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069025+0000", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4970000000000004e-06 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069042+0000", 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6515999999999999e-05 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069053+0000", 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0813e-05 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069155+0000", 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000102376 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069297+0000", 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000141769 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.069832+0000", 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053530299999999999 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:505 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [watch watch cookie 140139816826128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.305 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.15081460899999999, 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023993080000000002, 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 505 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:52.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070064+0000", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070071+0000", 2024-09-21T15:32:52.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6540000000000002e-06 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070087+0000", 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.641e-05 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070100+0000", 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2369e-05 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070216+0000", 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116913 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070341+0000", 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124232 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070585+0000", 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024396199999999999 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.071910+0000", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013251859999999999 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072263+0000", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035270100000000002 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072346+0000", 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3374999999999994e-05 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072465+0000", 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119073 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:531 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [watch unwatch cookie 140139816826128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.164549+0000", 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.056331089000000001, 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001735183, 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.311 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 531 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164549+0000", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164549+0000", 2024-09-21T15:32:52.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164546+0000", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164564+0000", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8142999999999999e-05 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164602+0000", 2024-09-21T15:32:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7577000000000003e-05 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164619+0000", 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164717+0000", 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8525999999999998e-05 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.164839+0000", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121809 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.165028+0000", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000189013 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.165907+0000", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00087906099999999999 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.166188+0000", 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000280346 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.166228+0000", 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0567e-05 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.166284+0000", 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6187999999999998e-05 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.317 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.317 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:52.318 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:94 trunc from 0x9a07478 to 0x809d94b 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:95 writesame 0x473dd77 thru 0x47471da data_size 0x2(0x9464 bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:97 write 0x5de3b80 thru 0x5de540a (0x188b bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:98 read 0x30f9f69 thru 0x31081e0 (0xe278 bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:100 write 0x69229f4 thru 0x693279d (0xfdaa bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:101 punch from 0x518fee7 to 0x519155f, (0x1678 bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:103 write 0x214a2e7 thru 0x214dcda (0x39f4 bytes) 2024-09-21T15:32:52.319 INFO:teuthology.orchestra.run.smithi107.stdout:104 punch from 0x339437f to 0x33962b9, (0x1f3a bytes) 2024-09-21T15:32:52.321 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:52.320+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:52.325 INFO:teuthology.orchestra.run.smithi107.stdout:105 write 0x857e1d3 thru 0x858734b (0x9179 bytes) 2024-09-21T15:32:52.334 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:52.334 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:52.334 INFO:teuthology.orchestra.run.smithi107.stdout:109 punch from 0x2a877cc to 0x2a93e1a, (0xc64e bytes) 2024-09-21T15:32:52.334 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xd7c446c 2024-09-21T15:32:52.345 INFO:teuthology.orchestra.run.smithi107.stdout:110 trunc from 0x858734c to 0xd7c446c 2024-09-21T15:32:52.362 INFO:teuthology.orchestra.run.smithi107.stdout:112 writesame 0xb83d69f thru 0xb84765d data_size 0x1(0x9fbf bytes) 2024-09-21T15:32:52.410 INFO:teuthology.orchestra.run.smithi107.stdout:113 writesame 0x191535d thru 0x191e191 data_size 0xf(0x8e35 bytes) 2024-09-21T15:32:52.418 INFO:teuthology.orchestra.run.smithi107.stdout:114 write 0x44080c9 thru 0x440eae0 (0x6a18 bytes) 2024-09-21T15:32:52.421 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:52.420+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:52.426 INFO:teuthology.orchestra.run.smithi107.stdout:115 write 0xcfac53b thru 0xcfac9eb (0x4b1 bytes) 2024-09-21T15:32:52.432 INFO:teuthology.orchestra.run.smithi107.stdout:116 punch from 0x2950da3 to 0x2954705, (0x3962 bytes) 2024-09-21T15:32:52.432 INFO:teuthology.orchestra.run.smithi107.stdout:118 read 0xc739c0d thru 0xc74094e (0x6d42 bytes) 2024-09-21T15:32:52.439 INFO:teuthology.orchestra.run.smithi107.stdout:119 trunc from 0xd7c446c to 0xaf6c278 2024-09-21T15:32:52.463 INFO:teuthology.orchestra.run.smithi107.stdout:122 write 0x80b45c thru 0x81153b (0x60e0 bytes) 2024-09-21T15:32:52.469 INFO:teuthology.orchestra.run.smithi107.stdout:123 write 0xa3f7c29 thru 0xa3fe447 (0x681f bytes) 2024-09-21T15:32:52.476 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:52.476 INFO:teuthology.orchestra.run.smithi107.stdout:125 read 0x7f24421 thru 0x7f2c4a3 (0x8083 bytes) 2024-09-21T15:32:52.476 INFO:teuthology.orchestra.run.smithi107.stdout:126 write 0x7dc25c8 thru 0x7dc6af0 (0x4529 bytes) 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:649 3.11 3:883ab1cb:::rbd_header.11911fd826fd:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.497103+0000", 2024-09-21T15:32:52.495 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00029959099999999998, 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000343751, 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 649 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497103+0000", 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497103+0000", 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497101+0000", 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497114+0000", 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2272999999999999e-05 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497143+0000", 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8949999999999999e-05 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497156+0000", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3653e-05 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497236+0000", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9789999999999993e-05 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.497368+0000", 2024-09-21T15:32:52.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000131674 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 1 2024-09-21T15:32:52.500 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:52.503 INFO:teuthology.orchestra.run.smithi107.stdout:128 writesame 0xc909fc6 thru 0xc9107ee data_size 0x1(0x6829 bytes) 2024-09-21T15:32:52.503 INFO:teuthology.orchestra.run.smithi107.stdout:129 writesame 0x15b835b thru 0x15bfbec data_size 0x3d(0x7892 bytes) 2024-09-21T15:32:52.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:32:52.511 INFO:teuthology.orchestra.run.smithi107.stdout:130 read 0x7621baa thru 0x762ada9 (0x9200 bytes) 2024-09-21T15:32:52.512 INFO:teuthology.orchestra.run.smithi107.stdout:131 punch from 0xa42c2ce to 0xa43aa22, (0xe754 bytes) 2024-09-21T15:32:52.513 INFO:teuthology.orchestra.run.smithi107.stdout:132 punch from 0x53bc22f to 0x53c8bee, (0xc9bf bytes) 2024-09-21T15:32:52.537 INFO:teuthology.orchestra.run.smithi107.stdout:133 trunc from 0xc9107ef to 0x377d84b 2024-09-21T15:32:52.555 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:52.555+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:52.570 INFO:teuthology.orchestra.run.smithi107.stdout:134 punch from 0x2eeb3ef to 0x2eec814, (0x1425 bytes) 2024-09-21T15:32:52.570 INFO:teuthology.orchestra.run.smithi107.stdout:136 write 0x2c20bf4 thru 0x2c248c8 (0x3cd5 bytes) 2024-09-21T15:32:52.575 INFO:teuthology.orchestra.run.smithi107.stdout:137 punch from 0x101fe6 to 0x10868a, (0x66a4 bytes) 2024-09-21T15:32:52.576 INFO:teuthology.orchestra.run.smithi107.stdout:138 trunc from 0x377d84b to 0x351281f 2024-09-21T15:32:52.620 INFO:teuthology.orchestra.run.smithi107.stdout:139 trunc from 0x351281f to 0xbcbc426 2024-09-21T15:32:52.622 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:52.620+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:52.636 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:52.636 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:52.636 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:52.637 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:52.637 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:52.650 INFO:teuthology.orchestra.run.smithi107.stdout:140 write 0xdf89e13 thru 0xdf922c5 (0x84b3 bytes) 2024-09-21T15:32:52.652 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:32:52.655 INFO:teuthology.orchestra.run.smithi107.stdout:141 write 0x5e548cb thru 0x5e61ef5 (0xd62b bytes) 2024-09-21T15:32:52.661 INFO:teuthology.orchestra.run.smithi107.stdout:142 write 0xdda1aff thru 0xdda3a16 (0x1f18 bytes) 2024-09-21T15:32:52.663 INFO:teuthology.orchestra.run.smithi107.stdout:143 write 0xb9eb641 thru 0xb9f0ec1 (0x5881 bytes) 2024-09-21T15:32:52.669 INFO:teuthology.orchestra.run.smithi107.stdout:144 writesame 0x7ecf154 thru 0x7ed7d2f data_size 0x2(0x8bdc bytes) 2024-09-21T15:32:52.685 INFO:teuthology.orchestra.run.smithi107.stdout:145 read 0x7fe5c6f thru 0x7fee951 (0x8ce3 bytes) 2024-09-21T15:32:52.685 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:52.685 INFO:teuthology.orchestra.run.smithi107.stdout:148 punch from 0xfbf0c1 to 0xfbfcda, (0xc19 bytes) 2024-09-21T15:32:52.685 INFO:teuthology.orchestra.run.smithi107.stdout:149 clone 3 order 24 su 8388608 sc 6 2024-09-21T15:32:52.723 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:52.720+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.031413499999999, 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011541064, 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.821 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 34 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787900+0000", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787899+0000", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787907+0000", 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6539999999999999e-06 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787920+0000", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2789000000000001e-05 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787938+0000", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8277000000000001e-05 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794653+0000", 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067151520000000003 2024-09-21T15:32:52.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794782+0000", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012917899999999999 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.794986+0000", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020385800000000001 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798337+0000", 2024-09-21T15:32:52.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033507490000000001 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799304+0000", 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00096694399999999996 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799386+0000", 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2188999999999995e-05 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799441+0000", 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5287000000000001e-05 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.031364800999999, 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011767003, 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.827 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 35 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787949+0000", 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.787948+0000", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788052+0000", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010343700000000001 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788065+0000", 2024-09-21T15:32:52.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3436e-05 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788073+0000", 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.795624+0000", 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075505720000000002 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.795858+0000", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000234451 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.796219+0000", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000360305 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799464+0000", 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0032457609999999998 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799544+0000", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9800000000000002e-05 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799631+0000", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7201999999999996e-05 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799716+0000", 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4560000000000004e-05 2024-09-21T15:32:52.832 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.031229659000001, 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011912222, 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.833 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 36 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788083+0000", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788104+0000", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1134000000000001e-05 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788115+0000", 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0135999999999999e-05 2024-09-21T15:32:52.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788122+0000", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5599999999999996e-06 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797472+0000", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093495780000000007 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797640+0000", 2024-09-21T15:32:52.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000168307 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797814+0000", 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174222 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799484+0000", 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001670019 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799875+0000", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039030400000000001 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799941+0000", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6632999999999997e-05 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799996+0000", 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5036e-05 2024-09-21T15:32:52.838 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.031180295999999, 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012487280999999999, 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788139+0000", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788148+0000", 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2860000000000005e-06 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788155+0000", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8469999999999998e-06 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798408+0000", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010253010999999999 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798541+0000", 2024-09-21T15:32:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013288899999999999 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798735+0000", 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000194351 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800035+0000", 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001299854 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800395+0000", 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036033000000000001 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800524+0000", 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012841600000000001 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800621+0000", 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6842999999999994e-05 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.934880782, 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011676723, 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 87 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884433+0000", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884432+0000", 2024-09-21T15:32:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884525+0000", 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3066000000000006e-05 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884539+0000", 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3253e-05 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884546+0000", 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8869999999999992e-06 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.891565+0000", 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0070186390000000001 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.891792+0000", 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022678800000000001 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.892195+0000", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000403407 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894459+0000", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022640289999999999 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895975+0000", 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001515366 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896052+0000", 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7169000000000006e-05 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896110+0000", 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7756000000000001e-05 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.93475647, 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012694309000000001, 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 88 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884556+0000", 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884563+0000", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5969999999999997e-06 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884572+0000", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2599999999999994e-06 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884582+0000", 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4299999999999995e-06 2024-09-21T15:32:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894521+0000", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099389100000000005 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894744+0000", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337400000000001 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895106+0000", 2024-09-21T15:32:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000361861 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896480+0000", 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013741000000000001 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897024+0000", 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054449900000000005 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897153+0000", 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000129016 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897252+0000", 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8091999999999995e-05 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755895701, 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011288622999999999, 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 110 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063418+0000", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063417+0000", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063430+0000", 2024-09-21T15:32:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.327e-05 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063449+0000", 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9556999999999999e-05 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063464+0000", 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.412e-05 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.069816+0000", 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063522939999999996 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.069925+0000", 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010952599999999999 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.070105+0000", 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017942500000000001 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072118+0000", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020134939999999998 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074588+0000", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024696879999999998 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074656+0000", 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8442999999999995e-05 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074707+0000", 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0219000000000001e-05 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755814378, 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011533918000000001, 2024-09-21T15:32:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 111 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063499+0000", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063498+0000", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063510+0000", 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.239e-05 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063530+0000", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9236999999999999e-05 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063544+0000", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4365999999999999e-05 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071126+0000", 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075817189999999998 2024-09-21T15:32:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071270+0000", 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014477500000000001 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.071468+0000", 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019767799999999999 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074797+0000", 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033287849999999999 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074910+0000", 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011262199999999999 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074971+0000", 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1456000000000002e-05 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075033+0000", 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2180000000000004e-05 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755756141999999, 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011602988, 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 112 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063558+0000", 2024-09-21T15:32:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063557+0000", 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063658+0000", 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010134899999999999 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063672+0000", 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3477e-05 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063680+0000", 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3760000000000008e-06 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072503+0000", 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0088231690000000005 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072642+0000", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000138512 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.072835+0000", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019289200000000001 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074891+0000", 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020561580000000002 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075060+0000", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016887200000000001 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075120+0000", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0078999999999999e-05 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075161+0000", 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0776999999999999e-05 2024-09-21T15:32:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755623149000002, 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013906523, 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063690+0000", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063696+0000", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8830000000000003e-06 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063705+0000", 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063712+0000", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1999999999999997e-06 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073726+0000", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010013986000000001 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073852+0000", 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125699 2024-09-21T15:32:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074033+0000", 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000181502 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077367+0000", 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033334409999999999 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077478+0000", 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110789 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077545+0000", 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7490000000000006e-05 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077597+0000", 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755593756, 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014011229, 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063719+0000", 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063725+0000", 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6069999999999998e-06 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063734+0000", 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063741+0000", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9129999999999997e-06 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075299+0000", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011557827999999999 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075415+0000", 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116066 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075607+0000", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019232099999999999 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077458+0000", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001851339 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077615+0000", 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015717899999999999 2024-09-21T15:32:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077677+0000", 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1467e-05 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077731+0000", 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4196000000000001e-05 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.755046657000001, 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015388028, 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 115 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064266+0000", 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064384+0000", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117656 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064400+0000", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5760000000000002e-05 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064409+0000", 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4429999999999992e-06 2024-09-21T15:32:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076158+0000", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011749226 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076294+0000", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136122 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076470+0000", 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017574600000000001 2024-09-21T15:32:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079442+0000", 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002971577 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079509+0000", 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7346000000000002e-05 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079582+0000", 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3412999999999997e-05 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079655+0000", 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2649000000000002e-05 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.754893438, 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015786732000000001, 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 116 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064419+0000", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064428+0000", 2024-09-21T15:32:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.986e-06 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064438+0000", 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8039999999999996e-06 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064445+0000", 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9789999999999996e-06 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077804+0000", 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013359207999999999 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077921+0000", 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117039 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078131+0000", 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000209395 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079687+0000", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001555758 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080102+0000", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000415547 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080163+0000", 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0705999999999998e-05 2024-09-21T15:32:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080207+0000", 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4202999999999997e-05 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.754860805, 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:32:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:32:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:32:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287748464, 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013241467, 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 136 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531570+0000", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0730000000000004e-06 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531579+0000", 2024-09-21T15:32:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1079999999999999e-06 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531585+0000", 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6730000000000003e-06 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536711+0000", 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0051255220000000004 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536808+0000", 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6739999999999999e-05 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536919+0000", 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110918 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539177+0000", 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002258262 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544733+0000", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055560599999999998 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544767+0000", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3918000000000003e-05 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544807+0000", 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9753999999999999e-05 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287722158000001, 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013385154, 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 137 2024-09-21T15:32:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531591+0000", 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531654+0000", 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2352000000000002e-05 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531660+0000", 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1569999999999997e-06 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531663+0000", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6480000000000001e-06 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537277+0000", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056140290000000004 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537387+0000", 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109131 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537569+0000", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182051 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540762+0000", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031933679999999998 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544872+0000", 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041098319999999999 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544927+0000", 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5470999999999998e-05 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544977+0000", 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9388999999999998e-05 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287645276999999, 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013408308000000001, 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 138 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531671+0000", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7659999999999999e-06 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531674+0000", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5499999999999999e-06 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531677+0000", 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0290000000000001e-06 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538356+0000", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066784920000000003 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538462+0000", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106572 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538622+0000", 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015995899999999999 2024-09-21T15:32:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541174+0000", 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025514589999999998 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545024+0000", 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0038503629999999999 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545055+0000", 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0278999999999999e-05 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545077+0000", 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2177000000000001e-05 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287633372999998, 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013559554, 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 139 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967296 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531682+0000", 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1280000000000002e-06 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531687+0000", 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4970000000000003e-06 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531694+0000", 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4560000000000003e-06 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539257+0000", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007562966 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539341+0000", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4319000000000004e-05 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539437+0000", 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5561000000000002e-05 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541183+0000", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017459629999999999 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545135+0000", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0039523300000000004 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545194+0000", 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8485e-05 2024-09-21T15:32:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545240+0000", 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6060999999999998e-05 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287605665000001, 2024-09-21T15:32:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013608026, 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 140 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531757+0000", 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9517e-05 2024-09-21T15:32:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531765+0000", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6960000000000005e-06 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531769+0000", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6380000000000002e-06 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539783+0000", 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080140839999999994 2024-09-21T15:32:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539891+0000", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010794900000000001 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540060+0000", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016991099999999999 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541649+0000", 2024-09-21T15:32:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015888370000000001 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545266+0000", 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036169309999999999 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545294+0000", 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8170999999999998e-05 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545316+0000", 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1642e-05 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.287540348, 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014580891, 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 141 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531776+0000", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531779+0000", 2024-09-21T15:32:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6179999999999999e-06 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531782+0000", 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541202+0000", 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094203670000000007 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:52.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541258+0000", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6283999999999998e-05 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541355+0000", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6038000000000004e-05 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.542023+0000", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000668656 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:52.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546250+0000", 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042266659999999996 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546307+0000", 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6805999999999998e-05 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546354+0000", 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.757e-05 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:52.938 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:52.938 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:52.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:32:52.939 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:52.936+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:52.986 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone2 from 0xdf922c6 (overlap 0x351281f) to 0x0 2024-09-21T15:32:53.040 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:53.039+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:53.085 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:53.085 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:53.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:32:53.121 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T15:32:53.168 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:53.168+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:53.242 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:53.239+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:53.264 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:32:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:32:53.264 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:53.280 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:32:53.342 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:53.339+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:53.414 INFO:teuthology.orchestra.run.smithi107.stdout:151 trunc from 0xdf922c6 to 0x64c75be 2024-09-21T15:32:53.443 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:53.440+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:53.459 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.924579842, 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055491940000000003, 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531931+0000", 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531944+0000", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3793e-05 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531977+0000", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2493000000000002e-05 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531990+0000", 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2799000000000001e-05 2024-09-21T15:32:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532542+0000", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055232999999999997 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532585+0000", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3040000000000001e-05 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537415+0000", 2024-09-21T15:32:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048296249999999997 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537482+0000", 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6987000000000001e-05 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.923163586000001, 2024-09-21T15:32:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060115560000000004, 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533347+0000", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533427+0000", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9798999999999999e-05 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533447+0000", 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0207000000000002e-05 2024-09-21T15:32:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533461+0000", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3413e-05 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534231+0000", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077009699999999997 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534286+0000", 2024-09-21T15:32:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5556999999999997e-05 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539301+0000", 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050144669999999999 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539360+0000", 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9472999999999998e-05 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.921282425000001, 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:32:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:32:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:32:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.920710349, 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:32:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'87)", 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.919572307999999, 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058430519999999996, 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536938+0000", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536962+0000", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.353e-05 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537014+0000", 2024-09-21T15:32:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2676000000000002e-05 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537041+0000", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538691+0000", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016501980000000001 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538726+0000", 2024-09-21T15:32:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5573e-05 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542722+0000", 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039951259999999999 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542783+0000", 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1465999999999997e-05 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:667 3.17 3:e8b0f5b3:::rbd_data.11911fd826fd.0000000000000044:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91427368600000003, 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056541559999999996, 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 667 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542235+0000", 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:32:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542257+0000", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2306999999999998e-05 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542307+0000", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9919000000000001e-05 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542331+0000", 2024-09-21T15:32:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3589999999999999e-05 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542459+0000", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128059 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000215751 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542976+0000", 2024-09-21T15:32:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030215199999999998 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545290+0000", 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023132449999999998 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547678+0000", 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023879309999999998 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547793+0000", 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011507599999999999 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547893+0000", 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100083 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:674 3.12 3:4e38c41e:::rbd_data.11911fd826fd.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91401678799999997, 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053068220000000001, 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 674 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542494+0000", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542502+0000", 2024-09-21T15:32:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8960000000000003e-06 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542523+0000", 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1365999999999998e-05 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542538+0000", 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4936999999999999e-05 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542642+0000", 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010399300000000001 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542830+0000", 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018741100000000001 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543162+0000", 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033192399999999999 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545215+0000", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020536310000000002 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547587+0000", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002371962 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547708+0000", 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012055199999999999 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547802+0000", 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4642999999999995e-05 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:897 3.1f 3:ff84ad69:::rbd_data.11911fd826fd.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.45232131599999997, 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061454999999999999, 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 897 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004190+0000", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004198+0000", 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7639999999999994e-06 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004247+0000", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8167999999999997e-05 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004260+0000", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004323+0000", 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3482e-05 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004424+0000", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101192 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004750+0000", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032567099999999998 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006929+0000", 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021786119999999999 2024-09-21T15:32:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010188+0000", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032598100000000001 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010282+0000", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3350000000000006e-05 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010337+0000", 2024-09-21T15:32:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4744999999999998e-05 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:924 3.1b 3:deb0c412:::rbd_data.11911fd826fd.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.44592662199999999, 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055231630000000002, 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 924 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010585+0000", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010593+0000", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8820000000000007e-06 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010609+0000", 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5614999999999999e-05 2024-09-21T15:32:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010616+0000", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0709999999999997e-06 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010681+0000", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5388999999999993e-05 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010793+0000", 2024-09-21T15:32:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111298 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.011051+0000", 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025864799999999999 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.012951+0000", 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001899235 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016004+0000", 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030537939999999999 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016063+0000", 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8619000000000002e-05 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016109+0000", 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5828999999999998e-05 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:946 3.1b 3:db55d07f:::rbd_data.11911fd826fd.000000000000001e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.44141614000000001, 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062820289999999997, 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 946 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015095+0000", 2024-09-21T15:32:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015102+0000", 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7009999999999999e-06 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015116+0000", 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.412e-05 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0429999999999999e-05 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015450+0000", 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031393000000000003 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015616+0000", 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016547900000000001 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021378+0000", 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057623930000000002 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:947 3.1b 3:de07fe9a:::rbd_data.11911fd826fd.000000000000001f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.016222+0000", 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.44028998800000002, 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00525896, 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 947 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016222+0000", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016222+0000", 2024-09-21T15:32:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016221+0000", 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016226+0000", 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.904e-06 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016239+0000", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.238e-05 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016245+0000", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4659999999999996e-06 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016297+0000", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1372999999999998e-05 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016432+0000", 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013496499999999999 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021481+0000", 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005049797 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:948 3.12 3:4a70ee63:::rbd_data.11911fd826fd.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.016266+0000", 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.44024600200000003, 2024-09-21T15:32:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051703210000000003, 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 948 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016266+0000", 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016266+0000", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016266+0000", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016269+0000", 2024-09-21T15:32:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7629999999999998e-06 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016276+0000", 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5440000000000004e-06 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016280+0000", 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4000000000000002e-06 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016329+0000", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8306000000000001e-05 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016409+0000", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0123999999999994e-05 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016630+0000", 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022098300000000001 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018852+0000", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022217999999999999 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021313+0000", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024612689999999999 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021383+0000", 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0234000000000006e-05 2024-09-21T15:32:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021437+0000", 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3525e-05 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:966 3.1f 3:fff846c7:::rbd_data.11911fd826fd.0000000000000023:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.024004+0000", 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.43250830699999998, 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051788440000000002, 2024-09-21T15:32:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 966 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024004+0000", 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024004+0000", 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024003+0000", 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024011+0000", 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9880000000000004e-06 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024025+0000", 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4034e-05 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024034+0000", 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4199999999999996e-06 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024099+0000", 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4453000000000001e-05 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024209+0000", 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011003 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.024498+0000", 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028986799999999998 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.026664+0000", 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021652849999999999 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029042+0000", 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023787449999999998 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029120+0000", 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7877e-05 2024-09-21T15:32:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029183+0000", 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2561e-05 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:975 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.029464+0000", 2024-09-21T15:32:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42704857600000001, 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051997479999999997, 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 975 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029464+0000", 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029464+0000", 2024-09-21T15:32:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029462+0000", 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029469+0000", 2024-09-21T15:32:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5509999999999996e-06 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029485+0000", 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5613000000000001e-05 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029493+0000", 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1720000000000007e-06 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029565+0000", 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2386999999999997e-05 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.029754+0000", 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018886400000000001 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034664+0000", 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049094539999999997 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:981 3.1 3:812dd4d6:::rbd_data.11911fd826fd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42422705100000002, 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073781309999999996, 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 981 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032284+0000", 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032291+0000", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032306+0000", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5162e-05 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032317+0000", 2024-09-21T15:32:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0641e-05 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032391+0000", 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3563e-05 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032531+0000", 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014023900000000001 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032886+0000", 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035541499999999998 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035001+0000", 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021150409999999998 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039546+0000", 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045443339999999997 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039615+0000", 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8936000000000005e-05 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039663+0000", 2024-09-21T15:32:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8866000000000003e-05 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:984 3.6 3:630b821a:::rbd_data.11911fd826fd.0000000000000033:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42375297899999997, 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006946069, 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032758+0000", 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032765+0000", 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4590000000000001e-06 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032780+0000", 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032789+0000", 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4339999999999998e-06 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032836+0000", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6891000000000002e-05 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032919+0000", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3238000000000005e-05 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033108+0000", 2024-09-21T15:32:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000188803 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035121+0000", 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020131900000000002 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044746530000000003 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039653+0000", 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6486000000000001e-05 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039705+0000", 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2935000000000001e-05 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:985 3.17 3:eca73502:::rbd_data.11911fd826fd.0000000000000034:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42334360599999998, 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0094186879999999997, 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033168+0000", 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033175+0000", 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1729999999999998e-06 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033194+0000", 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9383999999999999e-05 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033205+0000", 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0947e-05 2024-09-21T15:32:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033266+0000", 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0551999999999999e-05 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033373+0000", 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107155 2024-09-21T15:32:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033600+0000", 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022703799999999999 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035620+0000", 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020200050000000001 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042474+0000", 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068545359999999996 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042537+0000", 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2778000000000001e-05 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042587+0000", 2024-09-21T15:32:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0272999999999998e-05 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:982 3.10 e27/21 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head v 27'7, mlcod=26'5)", 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42323041700000003, 2024-09-21T15:32:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063444290000000004, 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033281+0000", 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033295+0000", 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3361e-05 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033306+0000", 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1268e-05 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033313+0000", 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6450000000000005e-06 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033365+0000", 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1208999999999997e-05 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033391+0000", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6353e-05 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039585+0000", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061938990000000001 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039626+0000", 2024-09-21T15:32:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:987 3.1f 3:fa7a7055:::rbd_data.11911fd826fd.0000000000000036:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.42152348899999997, 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077071170000000003, 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034988+0000", 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034996+0000", 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2360000000000004e-06 2024-09-21T15:32:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035015+0000", 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9072e-05 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035028+0000", 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2668e-05 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035078+0000", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0667999999999997e-05 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035162+0000", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3707000000000003e-05 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035389+0000", 2024-09-21T15:32:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022708 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.037465+0000", 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020759379999999998 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042600+0000", 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051350750000000002 2024-09-21T15:32:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042653+0000", 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3196999999999999e-05 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042696+0000", 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2800999999999999e-05 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:988 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.421317149, 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073861910000000003, 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-21T15:32:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035194+0000", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035203+0000", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2059999999999996e-06 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035222+0000", 2024-09-21T15:32:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9320000000000001e-05 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035233+0000", 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0793e-05 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035310+0000", 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6808000000000001e-05 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035479+0000", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016872999999999999 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035835+0000", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035607999999999999 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039777+0000", 2024-09-21T15:32:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039426979999999997 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042513+0000", 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002735786 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042550+0000", 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7211e-05 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042581+0000", 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1031000000000002e-05 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:53.574 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:53.575 INFO:teuthology.orchestra.run.smithi107.stdout:152 write 0xd1a215a thru 0xd1a4f25 (0x2dcc bytes) 2024-09-21T15:32:53.576 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:32:53.577 INFO:teuthology.orchestra.run.smithi107.stdout:153 trunc from 0xd1a4f26 to 0xaccf117 2024-09-21T15:32:53.577 INFO:teuthology.orchestra.run.smithi107.stdout:154 punch from 0x5e24a6a to 0x5e26cae, (0x2244 bytes) 2024-09-21T15:32:53.577 INFO:teuthology.orchestra.run.smithi107.stdout:155 write 0x6bc8294 thru 0x6bd7dfb (0xfb68 bytes) 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:156 writesame 0x66fe2b2 thru 0x670c8f9 data_size 0x4(0xe648 bytes) 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:157 punch from 0xec5ae2 to 0xecee2b, (0x9349 bytes) 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:158 punch from 0x32f3fee to 0x32f8f7d, (0x4f8f bytes) 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:160 trunc from 0xaccf117 to 0xafe3405 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:162 write 0xbad4b43 thru 0xbadd85d (0x8d1b bytes) 2024-09-21T15:32:53.578 INFO:teuthology.orchestra.run.smithi107.stdout:163 writesame 0x324389b thru 0x32537b1 data_size 0x13(0xff17 bytes) 2024-09-21T15:32:53.578 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:53.572+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:53.637 INFO:teuthology.orchestra.run.smithi107.stdout:164 write 0x8429ab1 thru 0x8429dd6 (0x326 bytes) 2024-09-21T15:32:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:169 punch from 0x9606d7b to 0x96146bd, (0xd942 bytes) 2024-09-21T15:32:53.644 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xdb9ecce 2024-09-21T15:32:53.648 INFO:teuthology.orchestra.run.smithi107.stdout:170 trunc from 0xbadd85e to 0xdb9ecce 2024-09-21T15:32:53.660 INFO:teuthology.orchestra.run.smithi107.stdout:172 writesame 0x19ba870 thru 0x19c530d data_size 0x1(0xaa9e bytes) 2024-09-21T15:32:53.676 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:53.675+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:53.756 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:53.756 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:53.761 INFO:teuthology.orchestra.run.smithi107.stdout:173 read 0x11b4a15 thru 0x11c0ddd (0xc3c9 bytes) 2024-09-21T15:32:53.762 INFO:teuthology.orchestra.run.smithi107.stdout:174 read 0xb492ada thru 0xb49a5fd (0x7b24 bytes) 2024-09-21T15:32:53.763 INFO:teuthology.orchestra.run.smithi107.stdout:175 read 0xeba53e thru 0xec6c7c (0xc73f bytes) 2024-09-21T15:32:53.764 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:53.764 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:53.764 INFO:teuthology.orchestra.run.smithi107.stdout:180 read 0x9412a26 thru 0x94190bb (0x6696 bytes) 2024-09-21T15:32:53.764 INFO:teuthology.orchestra.run.smithi107.stdout:181 punch from 0x4410b1f to 0x4417231, (0x6712 bytes) 2024-09-21T15:32:53.764 INFO:teuthology.orchestra.run.smithi107.stdout:182 read 0x5d7f28a thru 0x5d8e9a6 (0xf71d bytes) 2024-09-21T15:32:53.765 INFO:teuthology.orchestra.run.smithi107.stdout:183 read 0x93c0341 thru 0x93cae89 (0xab49 bytes) 2024-09-21T15:32:53.766 INFO:teuthology.orchestra.run.smithi107.stdout:184 flatten 2024-09-21T15:32:53.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:32:53.816 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:53.816+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:53.878 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:53.875+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:53.878 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:53.889 INFO:teuthology.orchestra.run.smithi107.stdout:186 trunc from 0xdb9ecce to 0x72adff0 2024-09-21T15:32:53.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:32:53.915 INFO:teuthology.orchestra.run.smithi107.stdout:189 read 0x606be0e thru 0x607b984 (0xfb77 bytes) 2024-09-21T15:32:53.916 INFO:teuthology.orchestra.run.smithi107.stdout:190 read 0xb692c7 thru 0xb78040 (0xed7a bytes) 2024-09-21T15:32:53.929 INFO:teuthology.orchestra.run.smithi107.stdout:191 trunc from 0x72adff0 to 0x3fcfb2 2024-09-21T15:32:53.952 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:32:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:32:53.952 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:53.968 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:32:53.979 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:53.975+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:53.980 INFO:teuthology.orchestra.run.smithi107.stdout:192 clone 4 order 20 su 32768 sc 15 2024-09-21T15:32:54.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:32:54.080 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:54.079+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:54.099 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3657699689999996, 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:32:54.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:32:54.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:621 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557257~27160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:54.104 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.673941422, 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031532890000000001, 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 621 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423627+0000", 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423663+0000", 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5459000000000001e-05 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423699+0000", 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6826000000000003e-05 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423716+0000", 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6654e-05 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423816+0000", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969000000000005e-05 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424070+0000", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025371399999999999 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424403+0000", 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033311799999999998 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426114+0000", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001711171 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426619+0000", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00050520199999999995 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426719+0000", 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9313000000000003e-05 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426784+0000", 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5629000000000005e-05 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:641 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write 490537~26655] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.623852855, 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003364204, 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 641 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473716+0000", 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473753+0000", 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6430000000000002e-05 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473790+0000", 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473827+0000", 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6779999999999997e-05 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473923+0000", 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6143000000000004e-05 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474178+0000", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000255374 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474426+0000", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024796799999999999 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.475922+0000", 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001495515 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476887+0000", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00096475599999999999 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476972+0000", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5562999999999994e-05 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.477084+0000", 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111319 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.115 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5553583469999999, 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:32:54.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:32:54.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:54.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:677 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:54.121 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5520882519999999, 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029472539999999998, 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 677 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545481+0000", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545497+0000", 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6555999999999999e-05 2024-09-21T15:32:54.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545535+0000", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7613000000000001e-05 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545555+0000", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.022e-05 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545683+0000", 2024-09-21T15:32:54.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012783599999999999 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545930+0000", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024724099999999999 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546209+0000", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000278428 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547436+0000", 2024-09-21T15:32:54.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012277900000000001 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548279+0000", 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00084254100000000004 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548374+0000", 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5501999999999995e-05 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548431+0000", 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6786999999999997e-05 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:687 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.545808061, 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028202320000000002, 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 687 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551763+0000", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551770+0000", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551789+0000", 2024-09-21T15:32:54.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9477e-05 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551833+0000", 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3760000000000001e-05 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554080+0000", 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002247125 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554308+0000", 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000228039 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554584+0000", 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027649400000000001 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:688 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:54.131 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5457257639999999, 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032222280000000002, 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 688 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551845+0000", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551852+0000", 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6499999999999999e-06 2024-09-21T15:32:54.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551870+0000", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7853e-05 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551882+0000", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2459999999999999e-05 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554681+0000", 2024-09-21T15:32:54.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002798189 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554810+0000", 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012915499999999999 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.555069+0000", 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025897799999999999 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:713 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.519894651, 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028307549999999999, 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 713 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577676+0000", 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577686+0000", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4930000000000004e-06 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577709+0000", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2923000000000001e-05 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577723+0000", 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4643e-05 2024-09-21T15:32:54.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580004+0000", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002280963 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580200+0000", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000195205 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580508+0000", 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030885400000000001 2024-09-21T15:32:54.139 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:716 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5198025450000001, 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034336240000000001, 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 716 2024-09-21T15:32:54.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577768+0000", 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577777+0000", 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4470000000000006e-06 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577796+0000", 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9769999999999999e-05 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577811+0000", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4686e-05 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580639+0000", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028279590000000001 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580881+0000", 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241955 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581203+0000", 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000322267 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5197471980000001, 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:54.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:32:54.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.148 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.519666763, 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:32:54.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:32:54.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.519621619, 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:32:54.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:32:54.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.519443391, 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:32:54.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:32:54.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.519369741, 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:32:54.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:32:54.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:32:54.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.1013629140000001, 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.169 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 25 entries 27'10 (0'0) error 3:5afb9d3f:::rbd_data.11911fd826fd.000000000000000c:head by client.4497.0:884 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 26'7)", 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.099320633, 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030724049999999998, 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998251+0000", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998264+0000", 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3871e-05 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998288+0000", 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3693000000000002e-05 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998307+0000", 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8612000000000001e-05 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998382+0000", 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5172999999999993e-05 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.001324+0000", 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029421310000000002 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 27 entries 27'12 (0'0) error 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head by client.4497.0:904 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 27'9)", 2024-09-21T15:32:54.173 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.090667775, 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003020773, 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:54.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006903+0000", 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006916+0000", 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2512e-05 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006939+0000", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3274000000000001e-05 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006955+0000", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5383000000000002e-05 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.007055+0000", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100079 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.009925+0000", 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028706690000000002 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:943 3.1a e27/21 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head v 27'13, mlcod=27'11)", 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.0824696149999999, 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034511770000000002, 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015101+0000", 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015110+0000", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6519999999999995e-06 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015127+0000", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7030000000000001e-05 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:32:54.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8440000000000004e-06 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015199+0000", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2786999999999994e-05 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015229+0000", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9927e-05 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018501+0000", 2024-09-21T15:32:54.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032721299999999998 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018554+0000", 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2970999999999998e-05 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.8 epoch 27/21 rep_tid 41 entries 27'3 (0'0) error 3:15312f39:::rbd_data.11911fd826fd.0000000000000020:head by client.4497.0:950 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.080373365, 2024-09-21T15:32:54.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037814699999999999, 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017205+0000", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7449999999999998e-06 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017223+0000", 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7626999999999999e-05 2024-09-21T15:32:54.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017241+0000", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.749e-05 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017302+0000", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1802000000000002e-05 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.020980+0000", 2024-09-21T15:32:54.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036779909999999998 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:988 3.6 e27/21 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head v 27'10, mlcod=26'6)", 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.061616256, 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036395920000000001, 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.185 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035955+0000", 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035964+0000", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035980+0000", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5855e-05 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035988+0000", 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8650000000000001e-06 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036064+0000", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6191000000000003e-05 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036088+0000", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3470000000000001e-05 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039519+0000", 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034318809999999999 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:54.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6165e-05 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:54.189 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:54.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:32:54.191 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:54.192 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:54.192 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:54.192 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:54.188+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:54.192 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:54.193 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:54.209 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:54.210 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:54.228 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:54.228 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:54.248 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:54.248 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:54.268 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:54.268 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:54.288 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:54.288 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:54.292 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:54.291+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:54.306 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:54.306 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:54.320 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:32:54.393 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:54.391+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:54.483 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:54.483 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:54.494 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:54.491+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:54.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:32:54.595 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:54.591+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:54.619 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:54.620 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:54.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:32:54.695 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:54.692+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:54.757 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:54.757 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:54.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:32:54.812 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:54.812+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:54.864 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:32:54.896 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:54.895+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:54.954 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:32:54.997 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:54.995+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:55.013 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T15:32:55.040 INFO:teuthology.orchestra.run.smithi107.stdout:193 write 0x6a86926 thru 0x6a8bcb0 (0x538b bytes) 2024-09-21T15:32:55.065 INFO:teuthology.orchestra.run.smithi107.stdout:194 write 0xbbe10e8 thru 0xbbec613 (0xb52c bytes) 2024-09-21T15:32:55.075 INFO:teuthology.orchestra.run.smithi107.stdout:195 read 0x97038ad thru 0x970c905 (0x9059 bytes) 2024-09-21T15:32:55.075 INFO:teuthology.orchestra.run.smithi107.stdout:196 write 0x5ddb4ff thru 0x5de2fbc (0x7abe bytes) 2024-09-21T15:32:55.083 INFO:teuthology.orchestra.run.smithi107.stdout:197 punch from 0xb7be199 to 0xb7c34fe, (0x5365 bytes) 2024-09-21T15:32:55.083 INFO:teuthology.orchestra.run.smithi107.stdout:198 writesame 0x26129ce thru 0x2615000 data_size 0xb(0x2633 bytes) 2024-09-21T15:32:55.090 INFO:teuthology.orchestra.run.smithi107.stdout:200 writesame 0x34a465d thru 0x34b0caa data_size 0x3(0xc64e bytes) 2024-09-21T15:32:55.090 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.090 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.098 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:55.095+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:55.105 INFO:teuthology.orchestra.run.smithi107.stdout:201 write 0xa0cbc83 thru 0xa0da985 (0xed03 bytes) 2024-09-21T15:32:55.107 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:32:55.115 INFO:teuthology.orchestra.run.smithi107.stdout:202 write 0xa24cf89 thru 0xa25c97c (0xf9f4 bytes) 2024-09-21T15:32:55.124 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.124 INFO:teuthology.orchestra.run.smithi107.stdout:205 read 0x23dfb6a thru 0x23ef794 (0xfc2b bytes) 2024-09-21T15:32:55.124 INFO:teuthology.orchestra.run.smithi107.stdout:206 punch from 0x8f9c56a to 0x8f9dc5c, (0x16f2 bytes) 2024-09-21T15:32:55.124 INFO:teuthology.orchestra.run.smithi107.stdout:208 writesame 0x155d517 thru 0x1561a86 data_size 0x58(0x4570 bytes) 2024-09-21T15:32:55.130 INFO:teuthology.orchestra.run.smithi107.stdout:210 write 0x7018e30 thru 0x7023ed6 (0xb0a7 bytes) 2024-09-21T15:32:55.135 INFO:teuthology.orchestra.run.smithi107.stdout:211 writesame 0x11e48d7 thru 0x11f3399 data_size 0x45(0xeac3 bytes) 2024-09-21T15:32:55.145 INFO:teuthology.orchestra.run.smithi107.stdout:212 writesame 0x61db38 thru 0x6238c1 data_size 0x27(0x5d8a bytes) 2024-09-21T15:32:55.159 INFO:teuthology.orchestra.run.smithi107.stdout:213 writesame 0xb6325a6 thru 0xb6347d2 data_size 0xd(0x222d bytes) 2024-09-21T15:32:55.168 INFO:teuthology.orchestra.run.smithi107.stdout:214 trunc from 0xbbec614 to 0xca63178 2024-09-21T15:32:55.177 INFO:teuthology.orchestra.run.smithi107.stdout:215 write 0x3cb3dd8 thru 0x3cbefc4 (0xb1ed bytes) 2024-09-21T15:32:55.183 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.184 INFO:teuthology.orchestra.run.smithi107.stdout:217 writesame 0xb69af08 thru 0xb6a1b1a data_size 0x1(0x6c13 bytes) 2024-09-21T15:32:55.199 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:55.196+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:55.202 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.202 INFO:teuthology.orchestra.run.smithi107.stdout:219 write 0x16b9478 thru 0x16c8659 (0xf1e2 bytes) 2024-09-21T15:32:55.208 INFO:teuthology.orchestra.run.smithi107.stdout:220 write 0x12ec1ce thru 0x12f2528 (0x635b bytes) 2024-09-21T15:32:55.216 INFO:teuthology.orchestra.run.smithi107.stdout:221 write 0x86ec5e0 thru 0x86ecfb1 (0x9d2 bytes) 2024-09-21T15:32:55.222 INFO:teuthology.orchestra.run.smithi107.stdout:223 punch from 0x9df4a9e to 0x9df6d40, (0x22a2 bytes) 2024-09-21T15:32:55.222 INFO:teuthology.orchestra.run.smithi107.stdout:224 write 0x6ad269f thru 0x6adbb40 (0x94a2 bytes) 2024-09-21T15:32:55.238 INFO:teuthology.orchestra.run.smithi107.stdout:225 trunc from 0xca63178 to 0x9b5c156 2024-09-21T15:32:55.252 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.252 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:227 read 0x11325eb thru 0x113c938 (0xa34e bytes) 2024-09-21T15:32:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:32:55.284 INFO:teuthology.orchestra.run.smithi107.stdout:229 write 0xb3b0299 thru 0xb3b3d8d (0x3af5 bytes) 2024-09-21T15:32:55.289 INFO:teuthology.orchestra.run.smithi107.stdout:230 write 0x62177dd thru 0x6225825 (0xe049 bytes) 2024-09-21T15:32:55.296 INFO:teuthology.orchestra.run.smithi107.stdout:231 writesame 0x12c26a7 thru 0x12c27ad data_size 0x1(0x107 bytes) 2024-09-21T15:32:55.302 INFO:teuthology.orchestra.run.smithi107.stdout:233 read 0xb4d9d4 thru 0xb5562d (0x7c5a bytes) 2024-09-21T15:32:55.315 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:55.315+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:55.323 INFO:teuthology.orchestra.run.smithi107.stdout:235 write 0xe3cedae thru 0xe3d700c (0x825f bytes) 2024-09-21T15:32:55.330 INFO:teuthology.orchestra.run.smithi107.stdout:236 writesame 0x6af07d8 thru 0x6af4702 data_size 0x1(0x3f2b bytes) 2024-09-21T15:32:55.339 INFO:teuthology.orchestra.run.smithi107.stdout:237 write 0x51cbeb1 thru 0x51cedcd (0x2f1d bytes) 2024-09-21T15:32:55.347 INFO:teuthology.orchestra.run.smithi107.stdout:238 read 0x4b643af thru 0x4b70c9f (0xc8f1 bytes) 2024-09-21T15:32:55.373 INFO:teuthology.orchestra.run.smithi107.stdout:240 trunc from 0xe3d700d to 0x386ddd9 2024-09-21T15:32:55.396 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.396 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.400 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:55.399+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:55.411 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.411 INFO:teuthology.orchestra.run.smithi107.stdout:242 read 0x1ddf92c thru 0x1de93b7 (0x9a8c bytes) 2024-09-21T15:32:55.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:32:55.448 INFO:teuthology.orchestra.run.smithi107.stdout:245 writesame 0xcd79ad0 thru 0xcd7b887 data_size 0xc(0x1db8 bytes) 2024-09-21T15:32:55.471 INFO:teuthology.orchestra.run.smithi107.stdout:246 trunc from 0xcd7b888 to 0x37e2692 2024-09-21T15:32:55.487 INFO:teuthology.orchestra.run.smithi107.stdout:248 read 0x91c000 thru 0x91de7d (0x1e7e bytes) 2024-09-21T15:32:55.487 INFO:teuthology.orchestra.run.smithi107.stdout:249 read 0x2bba42f thru 0x2bc8214 (0xdde6 bytes) 2024-09-21T15:32:55.502 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:55.500+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:55.526 INFO:teuthology.orchestra.run.smithi107.stdout:251 write 0xe2ce2db thru 0xe2d8d99 (0xaabf bytes) 2024-09-21T15:32:55.534 INFO:teuthology.orchestra.run.smithi107.stdout:252 read 0x52010e7 thru 0x5205ec9 (0x4de3 bytes) 2024-09-21T15:32:55.534 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xe72bdfe 2024-09-21T15:32:55.534 INFO:teuthology.orchestra.run.smithi107.stdout:253 trunc from 0xe2d8d9a to 0xe72bdfe 2024-09-21T15:32:55.545 INFO:teuthology.orchestra.run.smithi107.stdout:254 read 0x5e13dc5 thru 0x5e1d48d (0x96c9 bytes) 2024-09-21T15:32:55.546 INFO:teuthology.orchestra.run.smithi107.stdout:256 read 0x39116e8 thru 0x39119b7 (0x2d0 bytes) 2024-09-21T15:32:55.546 INFO:teuthology.orchestra.run.smithi107.stdout:259 writesame 0x6bf80d3 thru 0x6c030ce data_size 0xe(0xaffc bytes) 2024-09-21T15:32:55.564 INFO:teuthology.orchestra.run.smithi107.stdout:260 trunc from 0xe72bdfe to 0xa8ee1b0 2024-09-21T15:32:55.578 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.578 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:32:55.597 INFO:teuthology.orchestra.run.smithi107.stdout:261 trunc from 0xa8ee1b0 to 0xd80a2dd 2024-09-21T15:32:55.609 INFO:teuthology.orchestra.run.smithi107.stdout:263 write 0x38bfa14 thru 0x38c85fb (0x8be8 bytes) 2024-09-21T15:32:55.643 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:55.643+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:55.648 INFO:teuthology.orchestra.run.smithi107.stdout:265 trunc from 0xd80a2dd to 0x3813f43 2024-09-21T15:32:55.688 INFO:teuthology.orchestra.run.smithi107.stdout:268 write 0x4bda4e3 thru 0x4be15bc (0x70da bytes) 2024-09-21T15:32:55.699 INFO:teuthology.orchestra.run.smithi107.stdout:270 trunc from 0x4be15bd to 0x5dfd324 2024-09-21T15:32:55.703 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:55.700+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:55.710 INFO:teuthology.orchestra.run.smithi107.stdout:271 read 0x5a0a7b0 thru 0x5a15b50 (0xb3a1 bytes) 2024-09-21T15:32:55.720 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.720 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.734 INFO:teuthology.orchestra.run.smithi107.stdout:272 writesame 0x9f13b7b thru 0x9f1fcc1 data_size 0x1(0xc147 bytes) 2024-09-21T15:32:55.734 INFO:teuthology.orchestra.run.smithi107.stdout:273 read 0x5ac92d9 thru 0x5ac9566 (0x28e bytes) 2024-09-21T15:32:55.734 INFO:teuthology.orchestra.run.smithi107.stdout:275 read 0x20f6d07 thru 0x20f8dca (0x20c4 bytes) 2024-09-21T15:32:55.734 INFO:teuthology.orchestra.run.smithi107.stdout:276 write 0x57a960e thru 0x57b43a9 (0xad9c bytes) 2024-09-21T15:32:55.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:32:55.740 INFO:teuthology.orchestra.run.smithi107.stdout:277 read 0x7553bdc thru 0x75565fc (0x2a21 bytes) 2024-09-21T15:32:55.740 INFO:teuthology.orchestra.run.smithi107.stdout:278 write 0x7c1417 thru 0x7cfd07 (0xe8f1 bytes) 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:281 read 0x5867088 thru 0x586ff2b (0x8ea4 bytes) 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:282 punch from 0x8db4474 to 0x8dc0986, (0xc512 bytes) 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:283 read 0x67327bb thru 0x67358c2 (0x3108 bytes) 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:286 read 0x77e3a92 thru 0x77e7471 (0x39e0 bytes) 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.762 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.763 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.763 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.763 INFO:teuthology.orchestra.run.smithi107.stdout:298 write 0x9e37e97 thru 0x9e3d61a (0x5784 bytes) 2024-09-21T15:32:55.769 INFO:teuthology.orchestra.run.smithi107.stdout:300 read 0xa2ad2a thru 0xa2c1ae (0x1485 bytes) 2024-09-21T15:32:55.784 INFO:teuthology.orchestra.run.smithi107.stdout:301 write 0xb2ba370 thru 0xb2c4aa9 (0xa73a bytes) 2024-09-21T15:32:55.793 INFO:teuthology.orchestra.run.smithi107.stdout:303 read 0x2c0a66d thru 0x2c0eb4a (0x44de bytes) 2024-09-21T15:32:55.793 INFO:teuthology.orchestra.run.smithi107.stdout:305 punch from 0x329455b to 0x329d983, (0x9428 bytes) 2024-09-21T15:32:55.794 INFO:teuthology.orchestra.run.smithi107.stdout:307 read 0x7feca26 thru 0x7ff182c (0x4e07 bytes) 2024-09-21T15:32:55.794 INFO:teuthology.orchestra.run.smithi107.stdout:308 punch from 0xb0a5972 to 0xb0a661d, (0xcab bytes) 2024-09-21T15:32:55.794 INFO:teuthology.orchestra.run.smithi107.stdout:309 writesame 0x6b77062 thru 0x6b7a754 data_size 0x9(0x36f3 bytes) 2024-09-21T15:32:55.802 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.802 INFO:teuthology.orchestra.run.smithi107.stdout:314 read 0x76f30f thru 0x76f911 (0x603 bytes) 2024-09-21T15:32:55.802 INFO:teuthology.orchestra.run.smithi107.stdout:315 write 0x8468d99 thru 0x846ceba (0x4122 bytes) 2024-09-21T15:32:55.810 INFO:teuthology.orchestra.run.smithi107.stdout:317 trunc from 0xb2c4aaa to 0xbd01979 2024-09-21T15:32:55.821 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:55.824 INFO:teuthology.orchestra.run.smithi107.stdout:319 trunc from 0xbd01979 to 0xaaca10e 2024-09-21T15:32:55.827 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:55.827+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:55.839 INFO:teuthology.orchestra.run.smithi107.stdout:320 punch from 0x7843e0b to 0x784dba0, (0x9d95 bytes) 2024-09-21T15:32:55.839 INFO:teuthology.orchestra.run.smithi107.stdout:321 punch from 0x29a3e64 to 0x29abb32, (0x7cce bytes) 2024-09-21T15:32:55.839 INFO:teuthology.orchestra.run.smithi107.stdout:322 clone 5 order 24 su 524288 sc 5 2024-09-21T15:32:55.904 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:55.904+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:55.912 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:55.912 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:55.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:32:56.005 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:32:56.004+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:32:56.007 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone4 from 0xaaca10e (overlap 0x3fcfb2) to 0x2caafc 2024-09-21T15:32:56.097 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:56.097 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:56.105 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:56.103+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:56.112 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:32:56.207 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:56.204+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:56.210 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T15:32:56.240 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:56.240 INFO:teuthology.orchestra.run.smithi107.stdout:325 writesame 0x3dd83ec thru 0x3ddfd83 data_size 0xc(0x7998 bytes) 2024-09-21T15:32:56.287 INFO:teuthology.orchestra.run.smithi107.stdout:327 trunc from 0xaaca10e to 0x41278a5 2024-09-21T15:32:56.288 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:56.288 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:56.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:32:56.307 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:56.307+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:56.312 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:32:56.409 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:56.408+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:56.409 INFO:teuthology.orchestra.run.smithi107.stdout:328 trunc from 0x41278a5 to 0xde50c28 2024-09-21T15:32:56.418 INFO:teuthology.orchestra.run.smithi107.stdout:329 write 0x631ddae thru 0x6321096 (0x32e9 bytes) 2024-09-21T15:32:56.423 INFO:teuthology.orchestra.run.smithi107.stdout:330 read 0x5cbd929 thru 0x5ccc69d (0xed75 bytes) 2024-09-21T15:32:56.424 INFO:teuthology.orchestra.run.smithi107.stdout:331 write 0x3f26e49 thru 0x3f331e7 (0xc39f bytes) 2024-09-21T15:32:56.426 INFO:teuthology.orchestra.run.smithi107.stdout:332 read 0x91c01f2 thru 0x91c1363 (0x1172 bytes) 2024-09-21T15:32:56.427 INFO:teuthology.orchestra.run.smithi107.stdout:333 writesame 0xbe339d3 thru 0xbe3dac5 data_size 0x1(0xa0f3 bytes) 2024-09-21T15:32:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:334 punch from 0xb5dfc04 to 0xb5ec967, (0xcd63 bytes) 2024-09-21T15:32:56.466 INFO:teuthology.orchestra.run.smithi107.stdout:335 read 0x4ceb954 thru 0x4cfab86 (0xf233 bytes) 2024-09-21T15:32:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:337 read 0x4ae29a1 thru 0x4ae5603 (0x2c63 bytes) 2024-09-21T15:32:56.467 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:56.467 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:56.479 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:56.480 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:56.483 INFO:teuthology.orchestra.run.smithi107.stdout:339 trunc from 0xde50c28 to 0x25310c7 2024-09-21T15:32:56.484 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:56.496 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:32:56.499 INFO:teuthology.orchestra.run.smithi107.stdout:340 write 0xa7bc76 thru 0xa864b4 (0xa83f bytes) 2024-09-21T15:32:56.501 INFO:teuthology.orchestra.run.smithi107.stdout:341 punch from 0x135e4a1 to 0x136954a, (0xb0a9 bytes) 2024-09-21T15:32:56.503 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:56.509 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:56.508+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:56.512 INFO:teuthology.orchestra.run.smithi107.stdout:342 write 0x48dd170 thru 0x48ddbb7 (0xa48 bytes) 2024-09-21T15:32:56.514 INFO:teuthology.orchestra.run.smithi107.stdout:343 trunc from 0x48ddbb8 to 0x42129ce 2024-09-21T15:32:56.522 INFO:teuthology.orchestra.run.smithi107.stdout:344 write 0x3201b19 thru 0x3202126 (0x60e bytes) 2024-09-21T15:32:56.523 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:56.523 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:56.524 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:56.524 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:56.524 INFO:teuthology.orchestra.run.smithi107.stdout:349 write 0x235560 thru 0x236c24 (0x16c5 bytes) 2024-09-21T15:32:56.538 INFO:teuthology.orchestra.run.smithi107.stdout:350 write 0x97bb4c4 thru 0x97c442e (0x8f6b bytes) 2024-09-21T15:32:56.541 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:56.542 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:56.542 INFO:teuthology.orchestra.run.smithi107.stdout:351 punch from 0x394b3c2 to 0x3953432, (0x8070 bytes) 2024-09-21T15:32:56.542 INFO:teuthology.orchestra.run.smithi107.stdout:352 read 0x4d09966 thru 0x4d15f13 (0xc5ae bytes) 2024-09-21T15:32:56.542 INFO:teuthology.orchestra.run.smithi107.stdout:354 read 0x303cd18 thru 0x30433e1 (0x66ca bytes) 2024-09-21T15:32:56.545 INFO:teuthology.orchestra.run.smithi107.stdout:355 trunc from 0x97c442f to 0x775f2bb 2024-09-21T15:32:56.559 INFO:teuthology.orchestra.run.smithi107.stdout:358 trunc from 0x775f2bb to 0x2c57977 2024-09-21T15:32:56.560 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:56.560 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:56.575 INFO:teuthology.orchestra.run.smithi107.stdout:359 punch from 0xadb0bf to 0xae7c55, (0xcb96 bytes) 2024-09-21T15:32:56.576 INFO:teuthology.orchestra.run.smithi107.stdout:360 write 0xac8f30 thru 0xacee1d (0x5eee bytes) 2024-09-21T15:32:56.580 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:56.581 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:56.593 INFO:teuthology.orchestra.run.smithi107.stdout:363 writesame 0x5e12417 thru 0x5e211e3 data_size 0x1(0xedcd bytes) 2024-09-21T15:32:56.593 INFO:teuthology.orchestra.run.smithi107.stdout:364 punch from 0xaa0a53 to 0xab0860, (0xfe0d bytes) 2024-09-21T15:32:56.593 INFO:teuthology.orchestra.run.smithi107.stdout:365 read 0x28e97be thru 0x28f72fc (0xdb3f bytes) 2024-09-21T15:32:56.601 INFO:teuthology.orchestra.run.smithi107.stdout:366 write 0x66e7cc2 thru 0x66f3030 (0xb36f bytes) 2024-09-21T15:32:56.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:56.602 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:56.605 INFO:teuthology.orchestra.run.smithi107.stdout:367 read 0x1441eb9 thru 0x144b5ae (0x96f6 bytes) 2024-09-21T15:32:56.605 INFO:teuthology.orchestra.run.smithi107.stdout:369 clone 6 order 23 su 262144 sc 8 2024-09-21T15:32:56.610 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:56.607+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.874838961, 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072729969999999998, 2024-09-21T15:32:56.626 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751175+0000", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751188+0000", 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3774000000000001e-05 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751208+0000", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0173000000000002e-05 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751232+0000", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3059999999999999e-05 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751344+0000", 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011212199999999999 2024-09-21T15:32:56.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751354+0000", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9199999999999999e-06 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755656+0000", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043023430000000001 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755870+0000", 2024-09-21T15:32:56.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000213809 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.756264+0000", 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039450599999999998 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.757715+0000", 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001451232 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758252+0000", 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053677 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758350+0000", 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7275000000000006e-05 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758450+0000", 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100603 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:25 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [watch watch cookie 140139749721184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.8528021969999999, 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027829349999999998, 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 25 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773214+0000", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773212+0000", 2024-09-21T15:32:56.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773236+0000", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3326999999999999e-05 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773255+0000", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9862999999999999e-05 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773295+0000", 2024-09-21T15:32:56.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9879999999999998e-05 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773378+0000", 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2818999999999994e-05 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773511+0000", 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132669 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.773743+0000", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023192500000000001 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.774930+0000", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011868510000000001 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775664+0000", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00073456699999999999 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775799+0000", 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013473900000000001 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.775997+0000", 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019791899999999999 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:505 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [watch watch cookie 140139816826128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:56.638 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.555950406, 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023993080000000002, 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 505 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070066+0000", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070064+0000", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070071+0000", 2024-09-21T15:32:56.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6540000000000002e-06 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070087+0000", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.641e-05 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070100+0000", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2369e-05 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070216+0000", 2024-09-21T15:32:56.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116913 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070341+0000", 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124232 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.070585+0000", 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024396199999999999 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.071910+0000", 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013251859999999999 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072263+0000", 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035270100000000002 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072346+0000", 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3374999999999994e-05 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.072465+0000", 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119073 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:615 3.a 3:52632bac:::rbd_data.11911fd826fd.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.417433+0000", 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.2085835469999999, 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031415979999999998, 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 615 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417433+0000", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417433+0000", 2024-09-21T15:32:56.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417429+0000", 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417446+0000", 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6500000000000001e-05 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417489+0000", 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2935999999999997e-05 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417505+0000", 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6160000000000001e-05 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417569+0000", 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4207000000000004e-05 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.417714+0000", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014516299999999999 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.418153+0000", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043855499999999998 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.420460+0000", 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023073490000000002 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.420475+0000", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4834999999999999e-05 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.420542+0000", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7084999999999999e-05 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.420574+0000", 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2072000000000001e-05 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.649 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:771 3.10 3:0ff47219:::rbd_data.11911fd826fd.0000000000000059:head [set-alloc-hint object_size 1048576 write_size 1048576,write 870603~54827] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.661501+0000", 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9645145939999997, 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0028213399999999999, 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 771 2024-09-21T15:32:56.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661501+0000", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661501+0000", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661499+0000", 2024-09-21T15:32:56.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661628+0000", 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128663 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661649+0000", 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1732000000000001e-05 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661659+0000", 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0022e-05 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661724+0000", 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4539e-05 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661828+0000", 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010416099999999999 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.661943+0000", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114778 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.662934+0000", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00099064899999999991 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.664257+0000", 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013229190000000001 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.664294+0000", 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.786e-05 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.664323+0000", 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8461000000000001e-05 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:847 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.655 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.966104+0000", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6599122720000001, 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050013549999999999, 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 847 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966104+0000", 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966104+0000", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966100+0000", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966136+0000", 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5880000000000002e-05 2024-09-21T15:32:56.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966173+0000", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6813000000000001e-05 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966193+0000", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.016e-05 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966293+0000", 2024-09-21T15:32:56.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9271999999999994e-05 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.966305+0000", 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2434e-05 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.968894+0000", 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025888180000000001 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.968967+0000", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2912999999999998e-05 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.969135+0000", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000168288 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.969993+0000", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00085843499999999995 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.971036+0000", 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001042783 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.971077+0000", 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0824000000000001e-05 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.971105+0000", 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8104999999999999e-05 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:886 3.a 3:53e78312:::rbd_data.11911fd826fd.000000000000000e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.000749+0000", 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6252666410000001, 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042272689999999996, 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 886 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000749+0000", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000749+0000", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000747+0000", 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000758+0000", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1263e-05 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000783+0000", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4352e-05 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000796+0000", 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3633e-05 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000968+0000", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017139299999999999 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.001368+0000", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040070699999999998 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004977+0000", 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036082620000000001 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.665 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "pg_update_log_missing(3.1e epoch 27/21 rep_tid 32 entries 27'1 (0'0) error 3:7a27b9d6:::rbd_data.11911fd826fd.0000000000000010:head by client.4497.0:892 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.004395+0000", 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6216208449999998, 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0032503499999999999, 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004395+0000", 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004395+0000", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004395+0000", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004411+0000", 2024-09-21T15:32:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6135999999999999e-05 2024-09-21T15:32:56.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004460+0000", 2024-09-21T15:32:56.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9450000000000003e-05 2024-09-21T15:32:56.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004500+0000", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9406000000000002e-05 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005119+0000", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000619105 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.007646+0000", 2024-09-21T15:32:56.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025268489999999998 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:896 3.a 3:53e78312:::rbd_data.11911fd826fd.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.005658+0000", 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6203580610000001, 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041441020000000002, 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.671 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 896 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005658+0000", 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005658+0000", 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005655+0000", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005672+0000", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7116999999999999e-05 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005714+0000", 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.227e-05 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.005735+0000", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0707e-05 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.006250+0000", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051464900000000005 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.006487+0000", 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023741100000000001 2024-09-21T15:32:56.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.006794+0000", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00030667100000000001 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008302+0000", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001508049 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.009687+0000", 2024-09-21T15:32:56.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013854589999999999 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.009741+0000", 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3804000000000002e-05 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.009802+0000", 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1218999999999998e-05 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:913 3.10 3:0aee139d:::rbd_data.11919c13aaf6.000000000000000b:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.008452+0000", 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.61756419, 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0026339110000000001, 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 913 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008452+0000", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008452+0000", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008451+0000", 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008455+0000", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4740000000000001e-06 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008465+0000", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9070000000000002e-06 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.008469+0000", 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.34e-06 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.010620+0000", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021504570000000001 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.010789+0000", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016973100000000001 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.011086+0000", 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029649500000000003 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.680 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:928 3.10 3:0aee139d:::rbd_data.11919c13aaf6.000000000000000b:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.012662+0000", 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.61335418, 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023716800000000001, 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 928 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.681 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012662+0000", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012662+0000", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012661+0000", 2024-09-21T15:32:56.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012666+0000", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6859999999999997e-06 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012676+0000", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6069999999999997e-06 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012683+0000", 2024-09-21T15:32:56.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6869999999999994e-06 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.013379+0000", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00069552700000000004 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.013599+0000", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219788 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015034+0000", 2024-09-21T15:32:56.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014350389999999999 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:931 3.10 3:0aee139d:::rbd_data.11919c13aaf6.000000000000000b:5 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.012859+0000", 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6131575599999999, 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002491685, 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.685 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 931 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012859+0000", 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012859+0000", 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012858+0000", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012863+0000", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3199999999999999e-06 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012874+0000", 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1266e-05 2024-09-21T15:32:56.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012884+0000", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.713e-06 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015075+0000", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021911209999999999 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015215+0000", 2024-09-21T15:32:56.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000140035 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015350+0000", 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013512999999999999 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:932 3.10 3:0aee139d:::rbd_data.11919c13aaf6.000000000000000b:5 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.012892+0000", 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6131235930000001, 2024-09-21T15:32:56.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027830799999999998, 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 932 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012892+0000", 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012892+0000", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012892+0000", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012896+0000", 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.284e-06 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012906+0000", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7939999999999997e-06 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012912+0000", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2689999999999996e-06 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015392+0000", 2024-09-21T15:32:56.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024793229999999999 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015508+0000", 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116836 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.015676+0000", 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016722500000000001 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.693 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:935 3.a 3:52632bac:::rbd_data.11911fd826fd.000000000000001c:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.012966+0000", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.613049942, 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0054938720000000003, 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 935 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012966+0000", 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012966+0000", 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012965+0000", 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012971+0000", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4160000000000001e-06 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012984+0000", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2422000000000001e-05 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.012992+0000", 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1529999999999997e-06 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.013058+0000", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6124000000000006e-05 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.013159+0000", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100592 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.013399+0000", 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024020199999999999 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.014887+0000", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014877969999999999 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.018184+0000", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0032970550000000001 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.018339+0000", 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000155346 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.018460+0000", 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012098499999999999 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:972 3.10 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.029023+0000", 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.5969931289999999, 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053123930000000003, 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.699 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 972 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029023+0000", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029023+0000", 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029020+0000", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029032+0000", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1404e-05 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029059+0000", 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.764e-05 2024-09-21T15:32:56.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029076+0000", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.679e-05 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029220+0000", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014397699999999999 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.029521+0000", 2024-09-21T15:32:56.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00030058599999999998 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034335+0000", 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0048146580000000003 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:982 3.10 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.5913007750000001, 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0073293079999999997, 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.703 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 982 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:32:56.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034713+0000", 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034725+0000", 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2439999999999999e-05 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034761+0000", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6146000000000003e-05 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034781+0000", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9293999999999999e-05 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034849+0000", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7986999999999998e-05 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034960+0000", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011157 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.035187+0000", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022666599999999999 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.036202+0000", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001014741 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041899+0000", 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056968909999999999 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041978+0000", 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9405000000000003e-05 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.042045+0000", 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6587999999999993e-05 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:991 3.10 3:0ff47219:::rbd_data.11911fd826fd.0000000000000059:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.045250+0000", 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.5807663119999997, 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027362760000000002, 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 991 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045250+0000", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045250+0000", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045248+0000", 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:56.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045258+0000", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7820000000000005e-06 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045304+0000", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5927000000000003e-05 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045326+0000", 2024-09-21T15:32:56.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1800000000000001e-05 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045407+0000", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1258e-05 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045502+0000", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4707e-05 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045679+0000", 2024-09-21T15:32:56.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000177753 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.047780+0000", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021011490000000001 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.047805+0000", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4754000000000001e-05 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.047907+0000", 2024-09-21T15:32:56.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000102175 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.047986+0000", 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8681000000000001e-05 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1148 3.10 3:0e005eb6:::rbd_data.1191bb6b225c.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.425443+0000", 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.2005728410000001, 2024-09-21T15:32:56.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0026729620000000001, 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1148 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425443+0000", 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425443+0000", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425440+0000", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425455+0000", 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4997000000000001e-05 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425500+0000", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4486000000000001e-05 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425523+0000", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2926999999999999e-05 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425646+0000", 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122686 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.425998+0000", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035222699999999999 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.428116+0000", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021183959999999998 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1189 3.10 3:0e005eb6:::rbd_data.1191bb6b225c.0000000000000007:head [set-alloc-hint object_size 16777216 write_size 16777216,write 3965588~64360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.509072+0000", 2024-09-21T15:32:56.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.1169438, 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033798769999999999, 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1189 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509072+0000", 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509072+0000", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509069+0000", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509236+0000", 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016716800000000001 2024-09-21T15:32:56.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509271+0000", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4597000000000001e-05 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509291+0000", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9525999999999999e-05 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509415+0000", 2024-09-21T15:32:56.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012459299999999999 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509559+0000", 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014352899999999999 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.509791+0000", 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023241099999999999 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.511829+0000", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020375609999999998 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512300+0000", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00047119600000000001 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512384+0000", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4416e-05 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512452+0000", 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7892999999999995e-05 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1291 3.10 3:0e005eb6:::rbd_data.1191bb6b225c.0000000000000007:head [truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.892938+0000", 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.7330777130000001, 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025166479999999998, 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:56.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1291 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.892938+0000", 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.892938+0000", 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.892936+0000", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.892948+0000", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1909999999999999e-05 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.892981+0000", 2024-09-21T15:32:56.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.273e-05 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.893001+0000", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0523999999999999e-05 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.893143+0000", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014141499999999999 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.893390+0000", 2024-09-21T15:32:56.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024666199999999998 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.893749+0000", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035984299999999999 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.895054+0000", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013042539999999999 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.895215+0000", 2024-09-21T15:32:56.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161898 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.895362+0000", 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014625599999999999 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.895455+0000", 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3265999999999998e-05 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:56.729 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:56.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:32:56.731 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:56.728+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:56.831 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:56.831+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:56.861 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone5 from 0x66f3031 (overlap 0x25310c7) to 0x1dc0d6c 2024-09-21T15:32:56.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:32:56.893 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:32:56.893 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:32:56.893 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:32:56.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:32:56.912 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:56.912 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:56.912 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:32:56.912 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:56.926 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:32:56.936 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:56.936+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:56.999 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T15:32:57.010 INFO:teuthology.orchestra.run.smithi107.stdout:370 read 0x4204019 thru 0x420d35e (0x9346 bytes) 2024-09-21T15:32:57.011 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:57.011 INFO:teuthology.orchestra.run.smithi107.stdout:372 write 0x66b550 thru 0x678389 (0xce3a bytes) 2024-09-21T15:32:57.033 INFO:teuthology.orchestra.run.smithi107.stdout:373 write 0x13bc6fb thru 0x13bf89a (0x31a0 bytes) 2024-09-21T15:32:57.033 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:57.031+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:57.044 INFO:teuthology.orchestra.run.smithi107.stdout:374 write 0x6335660 thru 0x63409b4 (0xb355 bytes) 2024-09-21T15:32:57.054 INFO:teuthology.orchestra.run.smithi107.stdout:376 punch from 0x4113d49 to 0x4117048, (0x32ff bytes) 2024-09-21T15:32:57.054 INFO:teuthology.orchestra.run.smithi107.stdout:377 write 0x4553178 thru 0x455b9c9 (0x8852 bytes) 2024-09-21T15:32:57.056 INFO:teuthology.orchestra.run.smithi107.stdout:379 write 0x1f39811 thru 0x1f41f61 (0x8751 bytes) 2024-09-21T15:32:57.073 INFO:teuthology.orchestra.run.smithi107.stdout:380 writesame 0x1d55acd thru 0x1d637ad data_size 0x2b(0xdce1 bytes) 2024-09-21T15:32:57.097 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:57.097 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:32:57.097 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:32:57.097 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:32:57.097 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:57.099 INFO:teuthology.orchestra.run.smithi107.stdout:381 write 0xc33f314 thru 0xc34dcc1 (0xe9ae bytes) 2024-09-21T15:32:57.107 INFO:teuthology.orchestra.run.smithi107.stdout:382 read 0xa675a81 thru 0xa67d7b6 (0x7d36 bytes) 2024-09-21T15:32:57.107 INFO:teuthology.orchestra.run.smithi107.stdout:383 punch from 0x8ac3e66 to 0x8ad07ef, (0xc989 bytes) 2024-09-21T15:32:57.107 INFO:teuthology.orchestra.run.smithi107.stdout:384 read 0x1a34a63 thru 0x1a3dbfd (0x919b bytes) 2024-09-21T15:32:57.107 INFO:teuthology.orchestra.run.smithi107.stdout:385 read 0x15f2949 thru 0x15fd792 (0xae4a bytes) 2024-09-21T15:32:57.108 INFO:teuthology.orchestra.run.smithi107.stdout:387 write 0x85aba32 thru 0x85b3a48 (0x8017 bytes) 2024-09-21T15:32:57.113 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:32:57.113 INFO:teuthology.orchestra.run.smithi107.stdout:388 read 0x1e65634 thru 0x1e6d7dc (0x81a9 bytes) 2024-09-21T15:32:57.124 INFO:teuthology.orchestra.run.smithi107.stdout:389 write 0xdbfb0f2 thru 0xdbfdd23 (0x2c32 bytes) 2024-09-21T15:32:57.132 INFO:teuthology.orchestra.run.smithi107.stdout:392 write 0x1a7bb95 thru 0x1a83aa3 (0x7f0f bytes) 2024-09-21T15:32:57.136 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:57.132+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:57.145 INFO:teuthology.orchestra.run.smithi107.stdout:393 clone 7 order 25 su 2097152 sc 6 2024-09-21T15:32:57.235 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:57.235+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:57.251 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 30, 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:32:56.713352+0000", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:32:19.548914+0000", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5799936, 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:32:57.252 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "jewel", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:32:57.253 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:57.254 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:57.255 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:57.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:57.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:57.258 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 8, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 8, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.25, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:57.259 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:32:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.9900000095367432, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.9900000095367432, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.265 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:32:57.266 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "30", 2024-09-21T15:32:57.267 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 9, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 30, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:32:57.268 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:32:57.269 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.270 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.5, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.5, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.5, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.5, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 1, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 1 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:32:57.271 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:32:57.276 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.277 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 17, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.283 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:32:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.291 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.293 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.294 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.295 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.296 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.301 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.302 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.303 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:32:57.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:32:57.307 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.309 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.310 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.311 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:32:57.312 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:32:57.313 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 21, 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:32:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:32:57.315 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:32:57.316 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:32:57.317 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:57.318 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.319 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:32:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [], 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:32:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [], 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:32:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [ 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 2, 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 1 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:32:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:32:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:32:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:32:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:32:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:57.329 INFO:tasks.thrashosds.thrasher:j is {'epoch': 30, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:32:56.713352+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:32:19.548914+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 30, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 17, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:32:57.330 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.488926323000001, 2024-09-21T15:32:57.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011912222, 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 36 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788084+0000", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788083+0000", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788104+0000", 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1134000000000001e-05 2024-09-21T15:32:57.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788115+0000", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0135999999999999e-05 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788122+0000", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5599999999999996e-06 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797472+0000", 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093495780000000007 2024-09-21T15:32:57.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797640+0000", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000168307 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.797814+0000", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174222 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799484+0000", 2024-09-21T15:32:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001670019 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799875+0000", 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039030400000000001 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799941+0000", 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6632999999999997e-05 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.799996+0000", 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5036e-05 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.488876959999999, 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012487280999999999, 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788133+0000", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788139+0000", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1700000000000002e-06 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788148+0000", 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2860000000000005e-06 2024-09-21T15:32:57.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.788155+0000", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8469999999999998e-06 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798408+0000", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010253010999999999 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798541+0000", 2024-09-21T15:32:57.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013288899999999999 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.798735+0000", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000194351 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800035+0000", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001299854 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800395+0000", 2024-09-21T15:32:57.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036033000000000001 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800524+0000", 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012841600000000001 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.800621+0000", 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6842999999999994e-05 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.341 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.392453134, 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012694309000000001, 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 88 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884556+0000", 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884563+0000", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5969999999999997e-06 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884572+0000", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2599999999999994e-06 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884582+0000", 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4299999999999995e-06 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894521+0000", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099389100000000005 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894744+0000", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337400000000001 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895106+0000", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000361861 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896480+0000", 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013741000000000001 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897024+0000", 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054449900000000005 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897153+0000", 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000129016 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897252+0000", 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8091999999999995e-05 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.213319812999998, 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013906523, 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.347 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:32:57.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063690+0000", 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063696+0000", 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8830000000000003e-06 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063705+0000", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063712+0000", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1999999999999997e-06 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073726+0000", 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010013986000000001 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073852+0000", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125699 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074033+0000", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000181502 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077367+0000", 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033334409999999999 2024-09-21T15:32:57.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077478+0000", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110789 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077545+0000", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7490000000000006e-05 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077597+0000", 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:32:57.352 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.21329042, 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014011229, 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.353 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063719+0000", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063725+0000", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6069999999999998e-06 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063734+0000", 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:32:57.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063741+0000", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9129999999999997e-06 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075299+0000", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011557827999999999 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075415+0000", 2024-09-21T15:32:57.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116066 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075607+0000", 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019232099999999999 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077458+0000", 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001851339 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077615+0000", 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015717899999999999 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077677+0000", 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1467e-05 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077731+0000", 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4196000000000001e-05 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.212743321000001, 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015388028, 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 115 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.359 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064266+0000", 2024-09-21T15:32:57.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064384+0000", 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117656 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064400+0000", 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5760000000000002e-05 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064409+0000", 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4429999999999992e-06 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076158+0000", 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011749226 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076294+0000", 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136122 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076470+0000", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017574600000000001 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079442+0000", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002971577 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079509+0000", 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7346000000000002e-05 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079582+0000", 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3412999999999997e-05 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079655+0000", 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2649000000000002e-05 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.364 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.212590102, 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015786732000000001, 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 116 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064419+0000", 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064428+0000", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.986e-06 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064438+0000", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8039999999999996e-06 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064445+0000", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9789999999999996e-06 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077804+0000", 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013359207999999999 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077921+0000", 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117039 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078131+0000", 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000209395 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079687+0000", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001555758 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080102+0000", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000415547 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080163+0000", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0705999999999998e-05 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080207+0000", 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4202999999999997e-05 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.212557469, 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.370 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:32:57.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:32:57.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745445128, 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013241467, 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 136 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531570+0000", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0730000000000004e-06 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531579+0000", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1079999999999999e-06 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531585+0000", 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6730000000000003e-06 2024-09-21T15:32:57.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536711+0000", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0051255220000000004 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536808+0000", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6739999999999999e-05 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536919+0000", 2024-09-21T15:32:57.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110918 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539177+0000", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002258262 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544733+0000", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055560599999999998 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544767+0000", 2024-09-21T15:32:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3918000000000003e-05 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544807+0000", 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9753999999999999e-05 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745418822000001, 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013385154, 2024-09-21T15:32:57.381 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 137 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531591+0000", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531654+0000", 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2352000000000002e-05 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531660+0000", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1569999999999997e-06 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531663+0000", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6480000000000001e-06 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537277+0000", 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056140290000000004 2024-09-21T15:32:57.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537387+0000", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109131 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537569+0000", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182051 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540762+0000", 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031933679999999998 2024-09-21T15:32:57.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544872+0000", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041098319999999999 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544927+0000", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5470999999999998e-05 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544977+0000", 2024-09-21T15:32:57.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9388999999999998e-05 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745341941, 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013408308000000001, 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 138 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531671+0000", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7659999999999999e-06 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531674+0000", 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5499999999999999e-06 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531677+0000", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0290000000000001e-06 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538356+0000", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066784920000000003 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538462+0000", 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106572 2024-09-21T15:32:57.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538622+0000", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015995899999999999 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541174+0000", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025514589999999998 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545024+0000", 2024-09-21T15:32:57.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0038503629999999999 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545055+0000", 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0278999999999999e-05 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545077+0000", 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2177000000000001e-05 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745330036999999, 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013559554, 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 139 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967296 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531682+0000", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1280000000000002e-06 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531687+0000", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4970000000000003e-06 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531694+0000", 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4560000000000003e-06 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539257+0000", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007562966 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539341+0000", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4319000000000004e-05 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539437+0000", 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5561000000000002e-05 2024-09-21T15:32:57.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541183+0000", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017459629999999999 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545135+0000", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0039523300000000004 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545194+0000", 2024-09-21T15:32:57.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8485e-05 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545240+0000", 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6060999999999998e-05 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745302329000001, 2024-09-21T15:32:57.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013608026, 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 140 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531757+0000", 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9517e-05 2024-09-21T15:32:57.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531765+0000", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6960000000000005e-06 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531769+0000", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6380000000000002e-06 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539783+0000", 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080140839999999994 2024-09-21T15:32:57.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539891+0000", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010794900000000001 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540060+0000", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016991099999999999 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541649+0000", 2024-09-21T15:32:57.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015888370000000001 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545266+0000", 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036169309999999999 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545294+0000", 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8170999999999998e-05 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545316+0000", 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1642e-05 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 25.745237012, 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014580891, 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:32:57.404 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 141 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531776+0000", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531779+0000", 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6179999999999999e-06 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531782+0000", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541202+0000", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094203670000000007 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541258+0000", 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6283999999999998e-05 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541355+0000", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6038000000000004e-05 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.542023+0000", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000668656 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546250+0000", 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042266659999999996 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546307+0000", 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6805999999999998e-05 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546354+0000", 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.757e-05 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1190 3.11 3:8bc4bd9b:::rbd_data.1191bb6b225c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.409 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.7640845409999999, 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022028537000000001, 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1190 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512924+0000", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512937+0000", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3223e-05 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512966+0000", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9215999999999999e-05 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512980+0000", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4273e-05 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513106+0000", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012556000000000001 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513349+0000", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000243341 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.517115+0000", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003766138 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534677+0000", 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017561961000000001 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534756+0000", 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8382999999999998e-05 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534846+0000", 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0075999999999996e-05 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534954+0000", 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108446 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1220 3.7 e27/21 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head v 27'13, mlcod=27'11)", 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.6768515910000001, 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035610309, 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.415 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600156+0000", 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:32:57.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615711+0000", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015554596 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615756+0000", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5343000000000003e-05 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615775+0000", 2024-09-21T15:32:57.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8564e-05 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615881+0000", 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010644199999999999 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615921+0000", 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9133e-05 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635654+0000", 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019733581 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635769+0000", 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0001149 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1278 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:32:57.419 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.4924572980000002, 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.060972842999999999, 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1278 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784551+0000", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801400+0000", 2024-09-21T15:32:57.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01684892 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801431+0000", 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1581999999999997e-05 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801451+0000", 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.948e-05 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801589+0000", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013792500000000001 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801994+0000", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040495700000000001 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.802913+0000", 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00091958600000000001 2024-09-21T15:32:57.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.832473+0000", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029559932000000001 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845354+0000", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012880796999999999 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845448+0000", 2024-09-21T15:32:57.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3746000000000007e-05 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845526+0000", 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7972999999999994e-05 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1317 3.f 3:f0c8377b:::rbd_object_map.1191bb6b225c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.943079+0000", 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.3339309639999999, 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012287143, 2024-09-21T15:32:57.425 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1317 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943079+0000", 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943079+0000", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943079+0000", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943084+0000", 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8740000000000003e-06 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943093+0000", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6330000000000008e-06 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943100+0000", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8229999999999997e-06 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949859+0000", 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067589349999999998 2024-09-21T15:32:57.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950088+0000", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228691 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950793+0000", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070494799999999995 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.953979+0000", 2024-09-21T15:32:57.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031862219999999998 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955159+0000", 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011803880000000001 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955312+0000", 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015295799999999999 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955367+0000", 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4335000000000001e-05 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1318 3.f 3:f0c8377b:::rbd_object_map.1191bb6b225c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.943106+0000", 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.3339039480000001, 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012674523, 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1318 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943106+0000", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943106+0000", 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943106+0000", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943110+0000", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3100000000000002e-06 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943119+0000", 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6770000000000001e-06 2024-09-21T15:32:57.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.943126+0000", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.973e-06 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951548+0000", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084222889999999995 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951805+0000", 2024-09-21T15:32:57.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00025713800000000002 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.952574+0000", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00076860700000000004 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.954624+0000", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020499939999999999 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955707+0000", 2024-09-21T15:32:57.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001082871 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955750+0000", 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3103999999999999e-05 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955781+0000", 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1180000000000003e-05 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1323 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.3210162209999998, 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013965239000000001, 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1323 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955993+0000", 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956001+0000", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9629999999999998e-06 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956019+0000", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8454e-05 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956031+0000", 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2396e-05 2024-09-21T15:32:57.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956102+0000", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0840999999999995e-05 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956238+0000", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013587399999999999 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956409+0000", 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017057300000000001 2024-09-21T15:32:57.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.960773+0000", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043641009999999996 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969864+0000", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.009091132 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969927+0000", 2024-09-21T15:32:57.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2853000000000003e-05 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969959+0000", 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2583999999999998e-05 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:32:57.442 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:32:57.443 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:32:57.444 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:57.443+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:57.546 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:57.547+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:57.572 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:57.572 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:57.572 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:57.572 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:57.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:32:57.646 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:57.644+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:57.708 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:57.708 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:57.708 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:32:57.709 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:32:57.709 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:57.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:32:57.764 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:57.764+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:57.848 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:57.847+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:57.887 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:32:57.893 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:57.893 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:32:57.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:32:57.893 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:32:57.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:132 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'90, mlcod=15'85)", 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.359865426999999, 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055491940000000003, 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531933+0000", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531931+0000", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531944+0000", 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3793e-05 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531977+0000", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2493000000000002e-05 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.531990+0000", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2799000000000001e-05 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532542+0000", 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00055232999999999997 2024-09-21T15:32:57.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.532585+0000", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3040000000000001e-05 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537415+0000", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048296249999999997 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537482+0000", 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6987000000000001e-05 2024-09-21T15:32:57.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.358449171, 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060115560000000004, 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533347+0000", 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533427+0000", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9798999999999999e-05 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533447+0000", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0207000000000002e-05 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533461+0000", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3413e-05 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534231+0000", 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077009699999999997 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534286+0000", 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5556999999999997e-05 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539301+0000", 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050144669999999999 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539360+0000", 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9472999999999998e-05 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.35656801, 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.902 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:32:57.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:57.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.355995933999999, 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.908 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:32:57.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:138 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'96, mlcod=15'87)", 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.354857892999998, 2024-09-21T15:32:57.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058430519999999996, 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536940+0000", 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536938+0000", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536962+0000", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.353e-05 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537014+0000", 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2676000000000002e-05 2024-09-21T15:32:57.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537041+0000", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6333e-05 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538691+0000", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016501980000000001 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.538726+0000", 2024-09-21T15:32:57.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5573e-05 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542722+0000", 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039951259999999999 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542783+0000", 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1465999999999997e-05 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:667 3.17 3:e8b0f5b3:::rbd_data.11911fd826fd.0000000000000044:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3495592710000004, 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056541559999999996, 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 667 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542239+0000", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542235+0000", 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542257+0000", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2306999999999998e-05 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542307+0000", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9919000000000001e-05 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542331+0000", 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3589999999999999e-05 2024-09-21T15:32:57.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542459+0000", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128059 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000215751 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542976+0000", 2024-09-21T15:32:57.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030215199999999998 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545290+0000", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023132449999999998 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547678+0000", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023879309999999998 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547793+0000", 2024-09-21T15:32:57.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011507599999999999 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547893+0000", 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100083 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:674 3.12 3:4e38c41e:::rbd_data.11911fd826fd.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:57.922 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3493023730000004, 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053068220000000001, 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 674 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542496+0000", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542494+0000", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542502+0000", 2024-09-21T15:32:57.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8960000000000003e-06 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542523+0000", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1365999999999998e-05 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542538+0000", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4936999999999999e-05 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542642+0000", 2024-09-21T15:32:57.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010399300000000001 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542830+0000", 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018741100000000001 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543162+0000", 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033192399999999999 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545215+0000", 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020536310000000002 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547587+0000", 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002371962 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547708+0000", 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012055199999999999 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547802+0000", 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4642999999999995e-05 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:897 3.1f 3:ff84ad69:::rbd_data.11911fd826fd.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8876069009999998, 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061454999999999999, 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.928 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 897 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:32:57.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004190+0000", 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004198+0000", 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7639999999999994e-06 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004247+0000", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8167999999999997e-05 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004260+0000", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004323+0000", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3482e-05 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004424+0000", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101192 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004750+0000", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032567099999999998 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006929+0000", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021786119999999999 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010188+0000", 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032598100000000001 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010282+0000", 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3350000000000006e-05 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010337+0000", 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4744999999999998e-05 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:924 3.1b 3:deb0c412:::rbd_data.11911fd826fd.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8812122069999999, 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055231630000000002, 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 924 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.934 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010586+0000", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010585+0000", 2024-09-21T15:32:57.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010593+0000", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8820000000000007e-06 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010609+0000", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5614999999999999e-05 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010616+0000", 2024-09-21T15:32:57.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0709999999999997e-06 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010681+0000", 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5388999999999993e-05 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010793+0000", 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111298 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.011051+0000", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025864799999999999 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.012951+0000", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001899235 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016004+0000", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030537939999999999 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016063+0000", 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8619000000000002e-05 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.016109+0000", 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5828999999999998e-05 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:946 3.1b 3:db55d07f:::rbd_data.11911fd826fd.000000000000001e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:57.939 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8767017250000002, 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062820289999999997, 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 946 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015095+0000", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015102+0000", 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7009999999999999e-06 2024-09-21T15:32:57.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015116+0000", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.412e-05 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0429999999999999e-05 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015450+0000", 2024-09-21T15:32:57.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031393000000000003 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015616+0000", 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016547900000000001 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021378+0000", 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057623930000000002 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:981 3.1 3:812dd4d6:::rbd_data.11911fd826fd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8595126359999998, 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073781309999999996, 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 981 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032284+0000", 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032291+0000", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032306+0000", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5162e-05 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032317+0000", 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0641e-05 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032391+0000", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3563e-05 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032531+0000", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014023900000000001 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032886+0000", 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035541499999999998 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035001+0000", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021150409999999998 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039546+0000", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045443339999999997 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039615+0000", 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8936000000000005e-05 2024-09-21T15:32:57.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039663+0000", 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8866000000000003e-05 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:984 3.6 3:630b821a:::rbd_data.11911fd826fd.0000000000000033:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8590385640000004, 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006946069, 2024-09-21T15:32:57.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032758+0000", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032765+0000", 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4590000000000001e-06 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032780+0000", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032789+0000", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4339999999999998e-06 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032836+0000", 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6891000000000002e-05 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032919+0000", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3238000000000005e-05 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033108+0000", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000188803 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035121+0000", 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020131900000000002 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044746530000000003 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039653+0000", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6486000000000001e-05 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039705+0000", 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2935000000000001e-05 2024-09-21T15:32:57.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:985 3.17 3:eca73502:::rbd_data.11911fd826fd.0000000000000034:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8586291910000003, 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0094186879999999997, 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-21T15:32:57.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033168+0000", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033175+0000", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1729999999999998e-06 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033194+0000", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9383999999999999e-05 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033205+0000", 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0947e-05 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033266+0000", 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0551999999999999e-05 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033373+0000", 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107155 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033600+0000", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022703799999999999 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035620+0000", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020200050000000001 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042474+0000", 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068545359999999996 2024-09-21T15:32:57.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042537+0000", 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2778000000000001e-05 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042587+0000", 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0272999999999998e-05 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:982 3.10 e27/21 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head v 27'7, mlcod=26'5)", 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.858516002, 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063444290000000004, 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033281+0000", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033295+0000", 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3361e-05 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033306+0000", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1268e-05 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033313+0000", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6450000000000005e-06 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033365+0000", 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1208999999999997e-05 2024-09-21T15:32:57.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033391+0000", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6353e-05 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039585+0000", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061938990000000001 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039626+0000", 2024-09-21T15:32:57.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:987 3.1f 3:fa7a7055:::rbd_data.11911fd826fd.0000000000000036:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.8568090740000001, 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077071170000000003, 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034988+0000", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034996+0000", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2360000000000004e-06 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035015+0000", 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9072e-05 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035028+0000", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2668e-05 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035078+0000", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0667999999999997e-05 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035162+0000", 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3707000000000003e-05 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035389+0000", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022708 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.037465+0000", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020759379999999998 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042600+0000", 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051350750000000002 2024-09-21T15:32:57.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042653+0000", 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3196999999999999e-05 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042696+0000", 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2800999999999999e-05 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:988 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.856602734, 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073861910000000003, 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035194+0000", 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035203+0000", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2059999999999996e-06 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035222+0000", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9320000000000001e-05 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035233+0000", 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0793e-05 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035310+0000", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6808000000000001e-05 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035479+0000", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016872999999999999 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035835+0000", 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035607999999999999 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039777+0000", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039426979999999997 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042513+0000", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002735786 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042550+0000", 2024-09-21T15:32:57.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7211e-05 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042581+0000", 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1031000000000002e-05 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3169 3.b e30/21 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head v 30'28, mlcod=29'25)", 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.00702289, 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010485235000000001, 2024-09-21T15:32:57.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884773+0000", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884789+0000", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.556e-05 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884822+0000", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3189999999999999e-05 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884860+0000", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8166e-05 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884989+0000", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012823500000000001 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885053+0000", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4503000000000002e-05 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895205+0000", 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010151415 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895260+0000", 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5736000000000003e-05 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3170 3.17 3:eb82c48b:::rbd_data.1191ab8bc3c6.0000000000000001:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:32:57.980 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.0047875340000001, 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095184950000000001, 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3170 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887007+0000", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887024+0000", 2024-09-21T15:32:57.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.647e-05 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887072+0000", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8038999999999999e-05 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887096+0000", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3777e-05 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887229+0000", 2024-09-21T15:32:57.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133805 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887459+0000", 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229535 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887904+0000", 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044505 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892712+0000", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048083960000000004 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896348+0000", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036350990000000001 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896471+0000", 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012388300000000001 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896529+0000", 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7491000000000002e-05 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3172 3.b e30/21 3:d0ec678c:::rbd_data.1191ab8bc3c6.0000000000000003:head v 30'30, mlcod=29'25)", 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.99973615199999999, 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073246989999999996, 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.986 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892061+0000", 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:57.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892072+0000", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1301000000000001e-05 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892087+0000", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.504e-05 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892097+0000", 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969999999999993e-06 2024-09-21T15:32:57.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892204+0000", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106783 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892258+0000", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4014000000000002e-05 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899269+0000", 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070109120000000002 2024-09-21T15:32:57.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899386+0000", 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117793 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3173 3.1 3:84db22e7:::rbd_data.1191ab8bc3c6.0000000000000004:head [stat,truncate 6032748] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.99952566799999998, 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071419710000000004, 2024-09-21T15:32:57.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3173 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:57.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892269+0000", 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892286+0000", 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6826000000000001e-05 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892323+0000", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7292999999999997e-05 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892345+0000", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2186000000000001e-05 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892469+0000", 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000123566 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892666+0000", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019718900000000001 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893107+0000", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044130900000000003 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895184+0000", 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020761920000000001 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899255+0000", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040709980000000002 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899355+0000", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100543 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899414+0000", 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9119000000000001e-05 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.995 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:57.996 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:57.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:32:57.998 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:57.996+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:58.039 INFO:teuthology.orchestra.run.smithi107.stdout:leaving image image_client.0-clone6 intact 2024-09-21T15:32:58.098 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:58.095+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:58.167 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:58.167 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:58.168 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:58.168 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:58.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:32:58.228 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:32:58.228+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:32:58.300 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:58.299+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:58.306 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:58.306 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:58.306 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:32:58.306 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:32:58.306 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:58.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:32:58.349 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T15:32:58.400 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:58.399+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:58.430 INFO:teuthology.orchestra.run.smithi107.stdout:394 write 0xa83dbb4 thru 0xa844f0f (0x735c bytes) 2024-09-21T15:32:58.498 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.7621574350000007, 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:32:58.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:32:58.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:621 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557257~27160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.503 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.0703288879999997, 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031532890000000001, 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 621 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423627+0000", 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423663+0000", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5459000000000001e-05 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423699+0000", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6826000000000003e-05 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423716+0000", 2024-09-21T15:32:58.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6654e-05 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423816+0000", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969000000000005e-05 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424070+0000", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025371399999999999 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424403+0000", 2024-09-21T15:32:58.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033311799999999998 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426114+0000", 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001711171 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426619+0000", 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00050520199999999995 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426719+0000", 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9313000000000003e-05 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426784+0000", 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5629000000000005e-05 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:641 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write 490537~26655] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:58.509 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.0202403210000002, 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003364204, 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 641 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473716+0000", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473753+0000", 2024-09-21T15:32:58.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6430000000000002e-05 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473790+0000", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473827+0000", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6779999999999997e-05 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473923+0000", 2024-09-21T15:32:58.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6143000000000004e-05 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474178+0000", 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000255374 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474426+0000", 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024796799999999999 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.475922+0000", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001495515 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476887+0000", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00096475599999999999 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476972+0000", 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5562999999999994e-05 2024-09-21T15:32:58.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.477084+0000", 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111319 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9517458130000005, 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:32:58.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:32:58.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.520 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:677 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9484757180000001, 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029472539999999998, 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 677 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545484+0000", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545481+0000", 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545497+0000", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6555999999999999e-05 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545535+0000", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7613000000000001e-05 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545555+0000", 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.022e-05 2024-09-21T15:32:58.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545683+0000", 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012783599999999999 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.545930+0000", 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024724099999999999 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:32:58.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546209+0000", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000278428 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547436+0000", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012277900000000001 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548279+0000", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00084254100000000004 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548374+0000", 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5501999999999995e-05 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548431+0000", 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6786999999999997e-05 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:687 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.942195527, 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028202320000000002, 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 687 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551764+0000", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551763+0000", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551770+0000", 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4799999999999998e-06 2024-09-21T15:32:58.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551789+0000", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9477e-05 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551833+0000", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3760000000000001e-05 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554080+0000", 2024-09-21T15:32:58.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002247125 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554308+0000", 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000228039 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554584+0000", 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027649400000000001 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:688 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9421132300000004, 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032222280000000002, 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 688 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551845+0000", 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551852+0000", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6499999999999999e-06 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551870+0000", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7853e-05 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551882+0000", 2024-09-21T15:32:58.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2459999999999999e-05 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554681+0000", 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002798189 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554810+0000", 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012915499999999999 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.555069+0000", 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025897799999999999 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:713 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9162821169999997, 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028307549999999999, 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 713 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577678+0000", 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577676+0000", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577686+0000", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4930000000000004e-06 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577709+0000", 2024-09-21T15:32:58.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2923000000000001e-05 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577723+0000", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4643e-05 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580004+0000", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002280963 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580200+0000", 2024-09-21T15:32:58.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000195205 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580508+0000", 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030885400000000001 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:716 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9161900110000003, 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034336240000000001, 2024-09-21T15:32:58.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 716 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577768+0000", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577777+0000", 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4470000000000006e-06 2024-09-21T15:32:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577796+0000", 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9769999999999999e-05 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577811+0000", 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4686e-05 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580639+0000", 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028279590000000001 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580881+0000", 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241955 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581203+0000", 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000322267 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9161346640000003, 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.544 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.547 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9160542290000002, 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:32:58.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:32:58.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:32:58.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9160090849999998, 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:32:58.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:32:58.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9158308569999996, 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:32:58.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:32:58.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.9157572070000004, 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:32:58.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:58.565 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4977503800000003, 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:32:58.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:32:58.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 25 entries 27'10 (0'0) error 3:5afb9d3f:::rbd_data.11911fd826fd.000000000000000c:head by client.4497.0:884 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 26'7)", 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4957080989999998, 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030724049999999998, 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998251+0000", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998264+0000", 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3871e-05 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998288+0000", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3693000000000002e-05 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998307+0000", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8612000000000001e-05 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998382+0000", 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5172999999999993e-05 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.001324+0000", 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029421310000000002 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 27 entries 27'12 (0'0) error 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head by client.4497.0:904 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 27'9)", 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4870552410000002, 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003020773, 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006903+0000", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006916+0000", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2512e-05 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006939+0000", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3274000000000001e-05 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006955+0000", 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5383000000000002e-05 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.007055+0000", 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100079 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.009925+0000", 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028706690000000002 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:943 3.1a e27/21 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head v 27'13, mlcod=27'11)", 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4788570810000001, 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034511770000000002, 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015101+0000", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015110+0000", 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6519999999999995e-06 2024-09-21T15:32:58.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015127+0000", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7030000000000001e-05 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8440000000000004e-06 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015199+0000", 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2786999999999994e-05 2024-09-21T15:32:58.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015229+0000", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9927e-05 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018501+0000", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032721299999999998 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018554+0000", 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2970999999999998e-05 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.8 epoch 27/21 rep_tid 41 entries 27'3 (0'0) error 3:15312f39:::rbd_data.11911fd826fd.0000000000000020:head by client.4497.0:950 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.476760831, 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037814699999999999, 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017205+0000", 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7449999999999998e-06 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017223+0000", 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7626999999999999e-05 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017241+0000", 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.749e-05 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017302+0000", 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1802000000000002e-05 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.020980+0000", 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036779909999999998 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.584 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:988 3.6 e27/21 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head v 27'10, mlcod=26'6)", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.4580037219999999, 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036395920000000001, 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:32:58.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035955+0000", 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035964+0000", 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035980+0000", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5855e-05 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035988+0000", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8650000000000001e-06 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036064+0000", 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6191000000000003e-05 2024-09-21T15:32:58.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036088+0000", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3470000000000001e-05 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039519+0000", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034318809999999999 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6165e-05 2024-09-21T15:32:58.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:32:58.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:32:58.589 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:58.590 INFO:teuthology.orchestra.run.smithi107.stdout:396 writesame 0x702bf80 thru 0x703bec5 data_size 0x19(0xff46 bytes) 2024-09-21T15:32:58.591 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:32:58.591 INFO:teuthology.orchestra.run.smithi107.stdout:397 read 0x660ed40 thru 0x661296d (0x3c2e bytes) 2024-09-21T15:32:58.591 INFO:teuthology.orchestra.run.smithi107.stdout:399 punch from 0x12b74d3 to 0x12c0271, (0x8d9e bytes) 2024-09-21T15:32:58.591 INFO:teuthology.orchestra.run.smithi107.stdout:400 write 0x42c262c thru 0x42c7811 (0x51e6 bytes) 2024-09-21T15:32:58.592 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:32:58.588+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:32:58.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:32:58.624 INFO:teuthology.orchestra.run.smithi107.stdout:402 read 0x95fdf21 thru 0x960ce2c (0xef0c bytes) 2024-09-21T15:32:58.627 INFO:teuthology.orchestra.run.smithi107.stdout:403 writesame 0xdaed195 thru 0xdaeec47 data_size 0x1(0x1ab3 bytes) 2024-09-21T15:32:58.640 INFO:teuthology.orchestra.run.smithi107.stdout:407 write 0xd157dcb thru 0xd157df4 (0x2a bytes) 2024-09-21T15:32:58.647 INFO:teuthology.orchestra.run.smithi107.stdout:408 writesame 0xa0ae38d thru 0xa0b81a6 data_size 0x62(0x9e1a bytes) 2024-09-21T15:32:58.651 INFO:teuthology.orchestra.run.smithi107.stdout:409 read 0xc2c39 thru 0xd0b42 (0xdf0a bytes) 2024-09-21T15:32:58.652 INFO:teuthology.orchestra.run.smithi107.stdout:410 writesame 0x454642 thru 0x45ad6a data_size 0x1(0x6729 bytes) 2024-09-21T15:32:58.669 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.669 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.676 INFO:teuthology.orchestra.run.smithi107.stdout:414 trunc from 0xdbfdd24 to 0x9d8eb46 2024-09-21T15:32:58.691 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:58.687+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:58.774 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:32:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:32:58.774 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:32:58.774 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:32:58.779 INFO:teuthology.orchestra.run.smithi107.stdout:416 write 0xbb76595 thru 0xbb81c15 (0xb681 bytes) 2024-09-21T15:32:58.783 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.783 INFO:teuthology.orchestra.run.smithi107.stdout:419 write 0x3ebaf96 thru 0x3ec9684 (0xe6ef bytes) 2024-09-21T15:32:58.787 INFO:teuthology.orchestra.run.smithi107.stdout:420 punch from 0x8123fcd to 0x8125d8f, (0x1dc2 bytes) 2024-09-21T15:32:58.787 INFO:teuthology.orchestra.run.smithi107.stdout:421 trunc from 0xbb81c16 to 0xb95a865 2024-09-21T15:32:58.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:32:58.792 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:58.791+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:58.793 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:32:58.794 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:32:58.796 INFO:teuthology.orchestra.run.smithi107.stdout:422 writesame 0x83c8a96 thru 0x83d5da6 data_size 0x1f(0xd311 bytes) 2024-09-21T15:32:58.799 INFO:teuthology.orchestra.run.smithi107.stdout:423 writesame 0x8237f31 thru 0x8240c4a data_size 0x1(0x8d1a bytes) 2024-09-21T15:32:58.813 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:32:58.813 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:32:58.818 INFO:teuthology.orchestra.run.smithi107.stdout:424 write 0x7cb02aa thru 0x7cb497a (0x46d1 bytes) 2024-09-21T15:32:58.821 INFO:teuthology.orchestra.run.smithi107.stdout:427 punch from 0x6a9cf63 to 0x6aab49c, (0xe539 bytes) 2024-09-21T15:32:58.831 INFO:teuthology.orchestra.run.smithi107.stdout:428 trunc from 0xb95a865 to 0x56e00c9 2024-09-21T15:32:58.837 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:32:58.837 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:32:58.847 INFO:teuthology.orchestra.run.smithi107.stdout:430 punch from 0xa3404c to 0xa3d9c9, (0x997d bytes) 2024-09-21T15:32:58.847 INFO:teuthology.orchestra.run.smithi107.stdout:431 write 0x50a4cb1 thru 0x50af6ae (0xa9fe bytes) 2024-09-21T15:32:58.856 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:32:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:32:58.860 INFO:teuthology.orchestra.run.smithi107.stdout:432 trunc from 0x56e00c9 to 0xc3bfbe2 2024-09-21T15:32:58.869 INFO:teuthology.orchestra.run.smithi107.stdout:434 read 0x4db1059 thru 0x4dbe5d4 (0xd57c bytes) 2024-09-21T15:32:58.870 INFO:teuthology.orchestra.run.smithi107.stdout:435 write 0x9c3b0e5 thru 0x9c3d435 (0x2351 bytes) 2024-09-21T15:32:58.872 INFO:teuthology.orchestra.run.smithi107.stdout:437 writesame 0x83fff06 thru 0x8404518 data_size 0x1(0x4613 bytes) 2024-09-21T15:32:58.880 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:32:58.880 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:32:58.882 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.886 INFO:teuthology.orchestra.run.smithi107.stdout:439 trunc from 0xc3bfbe2 to 0xa320391 2024-09-21T15:32:58.893 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:58.891+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:58.900 INFO:teuthology.orchestra.run.smithi107.stdout:440 punch from 0xc6e731 to 0xc744f3, (0x5dc2 bytes) 2024-09-21T15:32:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:442 read 0x1ccd96a thru 0x1cdb64b (0xdce2 bytes) 2024-09-21T15:32:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.901 INFO:teuthology.orchestra.run.smithi107.stdout:444 read 0x1e2cd3e thru 0x1e32313 (0x55d6 bytes) 2024-09-21T15:32:58.902 INFO:teuthology.orchestra.run.smithi107.stdout:445 read 0x4852df7 thru 0x4856a7a (0x3c84 bytes) 2024-09-21T15:32:58.902 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:32:58.903 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:32:58.903 INFO:teuthology.orchestra.run.smithi107.stdout:446 read 0x9f9df8 thru 0x9fa5a3 (0x7ac bytes) 2024-09-21T15:32:58.920 INFO:teuthology.orchestra.run.smithi107.stdout:447 trunc from 0xa320391 to 0x817445 2024-09-21T15:32:58.921 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:58.921 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:58.936 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:32:58.936 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:32:58.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:32:58.958 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:32:58.958 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:32:58.962 INFO:teuthology.orchestra.run.smithi107.stdout:449 write 0xa3a461c thru 0xa3abc92 (0x7677 bytes) 2024-09-21T15:32:58.966 INFO:teuthology.orchestra.run.smithi107.stdout:450 write 0x734e1f5 thru 0x735d1cf (0xefdb bytes) 2024-09-21T15:32:58.974 INFO:teuthology.orchestra.run.smithi107.stdout:452 trunc from 0xa3abc93 to 0x6919951 2024-09-21T15:32:58.985 INFO:teuthology.orchestra.run.smithi107.stdout:453 write 0x26bfef2 thru 0x26cbc20 (0xbd2f bytes) 2024-09-21T15:32:58.989 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.989 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:58.993 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:32:58.991+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:32:59.039 INFO:teuthology.orchestra.run.smithi107.stdout:457 trunc from 0x6919951 to 0x3a0cff0 2024-09-21T15:32:59.039 INFO:teuthology.orchestra.run.smithi107.stdout:458 write 0xbea20df thru 0xbeaecb6 (0xcbd8 bytes) 2024-09-21T15:32:59.039 INFO:teuthology.orchestra.run.smithi107.stdout:459 punch from 0xbe61965 to 0xbe63c25, (0x22c0 bytes) 2024-09-21T15:32:59.039 INFO:teuthology.orchestra.run.smithi107.stdout:460 writesame 0x8403ce3 thru 0x840f981 data_size 0x1(0xbc9f bytes) 2024-09-21T15:32:59.053 INFO:teuthology.orchestra.run.smithi107.stdout:461 punch from 0xa7b2dbf to 0xa7c11ef, (0xe430 bytes) 2024-09-21T15:32:59.065 INFO:teuthology.orchestra.run.smithi107.stdout:462 write 0xe5ad54d thru 0xe5ad9e8 (0x49c bytes) 2024-09-21T15:32:59.071 INFO:teuthology.orchestra.run.smithi107.stdout:463 write 0xde69fe2 thru 0xde6b153 (0x1172 bytes) 2024-09-21T15:32:59.076 INFO:teuthology.orchestra.run.smithi107.stdout:464 write 0x8ec18ee thru 0x8ec6134 (0x4847 bytes) 2024-09-21T15:32:59.078 INFO:teuthology.orchestra.run.smithi107.stdout:466 writesame 0xd25f41f thru 0xd26a496 data_size 0x4(0xb078 bytes) 2024-09-21T15:32:59.092 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.092 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.094 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:59.091+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:59.095 INFO:teuthology.orchestra.run.smithi107.stdout:467 write 0x194686 thru 0x1a0f9c (0xc917 bytes) 2024-09-21T15:32:59.099 INFO:teuthology.orchestra.run.smithi107.stdout:468 writesame 0x5407fa0 thru 0x541034e data_size 0x11(0x83af bytes) 2024-09-21T15:32:59.112 INFO:teuthology.orchestra.run.smithi107.stdout:469 trunc from 0xe5ad9e9 to 0x8951ea5 2024-09-21T15:32:59.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:32:59.154 INFO:teuthology.orchestra.run.smithi107.stdout:470 trunc from 0x8951ea5 to 0xb2e6d1 2024-09-21T15:32:59.166 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:59.166 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:32:59.182 INFO:teuthology.orchestra.run.smithi107.stdout:474 writesame 0x3af8c54 thru 0x3af9fdb data_size 0xa(0x1388 bytes) 2024-09-21T15:32:59.184 INFO:teuthology.orchestra.run.smithi107.stdout:475 read 0x131eac2 thru 0x132beb5 (0xd3f4 bytes) 2024-09-21T15:32:59.185 INFO:teuthology.orchestra.run.smithi107.stdout:479 clone 8 order 22 su 524288 sc 7 2024-09-21T15:32:59.195 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:59.195+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:59.273 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.273 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:32:59.328 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:32:59.328+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:32:59.398 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:59.395+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:32:59.499 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:32:59.499+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:32:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:32:59.639 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:32:59.639+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:32:59.700 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:59.699+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:59.723 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.723 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.737 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:32:59.823 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:59.823+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:59.898 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:32:59.899 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:32:59.902 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:32:59.899+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:32:59.915 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:32:59.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:00.044 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:00.044+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:00.052 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone7 2024-09-21T15:33:00.083 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.083 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.098 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:00.104 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:00.100+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:00.173 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T15:33:00.205 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:00.204+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:00.215 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.215 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.229 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:00.306 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:00.304+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:00.348 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.348 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:00.407 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:00.407+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:00.481 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.481 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:00.508 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:00.504+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:00.608 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:00.607+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:00.618 INFO:teuthology.orchestra.run.smithi107.stdout:480 trunc from 0x3af9fdc to 0x536277a 2024-09-21T15:33:00.653 INFO:teuthology.orchestra.run.smithi107.stdout:481 write 0xa122213 thru 0xa12cb5c (0xa94a bytes) 2024-09-21T15:33:00.660 INFO:teuthology.orchestra.run.smithi107.stdout:482 writesame 0x2042fc3 thru 0x2046c3d data_size 0xd(0x3c7b bytes) 2024-09-21T15:33:00.670 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.670 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:00.689 INFO:teuthology.orchestra.run.smithi107.stdout:484 write 0xeabd970 thru 0xeacc60d (0xec9e bytes) 2024-09-21T15:33:00.695 INFO:teuthology.orchestra.run.smithi107.stdout:485 writesame 0x22556e2 thru 0x2258380 data_size 0x1(0x2c9f bytes) 2024-09-21T15:33:00.705 INFO:teuthology.orchestra.run.smithi107.stdout:487 writesame 0xa70de9b thru 0xa71cdca data_size 0xac(0xef30 bytes) 2024-09-21T15:33:00.709 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:00.708+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:00.711 INFO:teuthology.orchestra.run.smithi107.stdout:489 read 0xa6b618b thru 0xa6becc8 (0x8b3e bytes) 2024-09-21T15:33:00.711 INFO:teuthology.orchestra.run.smithi107.stdout:491 read 0xd127d76 thru 0xd12bbff (0x3e8a bytes) 2024-09-21T15:33:00.712 INFO:teuthology.orchestra.run.smithi107.stdout:494 read 0x4c78b8a thru 0x4c81f96 (0x940d bytes) 2024-09-21T15:33:00.717 INFO:teuthology.orchestra.run.smithi107.stdout:495 trunc from 0xeacc60e to 0xbddbee0 2024-09-21T15:33:00.746 INFO:teuthology.orchestra.run.smithi107.stdout:497 trunc from 0xbddbee0 to 0x51829b8 2024-09-21T15:33:00.773 INFO:teuthology.orchestra.run.smithi107.stdout:498 write 0x4ced07 thru 0x4d2f7c (0x4276 bytes) 2024-09-21T15:33:00.781 INFO:teuthology.orchestra.run.smithi107.stdout:499 read 0x18cce90 thru 0x18cda26 (0xb97 bytes) 2024-09-21T15:33:00.781 INFO:teuthology.orchestra.run.smithi107.stdout:500 clone 9 order 20 su 131072 sc 2 2024-09-21T15:33:00.810 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:00.807+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:00.856 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:00.856 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:00.870 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:33:00.911 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:00.907+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:00.979 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:00.980 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone8 from 0x51829b8 (overlap 0x3af9fdc) to 0x2f2e649 2024-09-21T15:33:01.001 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:01.001 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:01.021 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:01.024 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:01.023+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:01.103 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T15:33:01.113 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:01.111+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:01.128 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:01.141 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:01.141 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:01.146 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:01.146 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:01.155 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:01.162 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:01.163 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:01.179 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:01.181 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:01.196 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:01.213 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:01.213 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:01.213 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:01.211+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:01.216 INFO:teuthology.orchestra.run.smithi107.stdout:502 trunc from 0x51829b8 to 0x1ffac4 2024-09-21T15:33:01.232 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:01.232 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:01.252 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:01.252 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:01.255 INFO:teuthology.orchestra.run.smithi107.stdout:503 writesame 0xb9f49c1 thru 0xb9fbde2 data_size 0xf(0x7422 bytes) 2024-09-21T15:33:01.255 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.255 INFO:teuthology.orchestra.run.smithi107.stdout:505 punch from 0x7729e21 to 0x772e46b, (0x464a bytes) 2024-09-21T15:33:01.255 INFO:teuthology.orchestra.run.smithi107.stdout:506 read 0x75a5cef thru 0x75afcea (0x9ffc bytes) 2024-09-21T15:33:01.262 INFO:teuthology.orchestra.run.smithi107.stdout:508 writesame 0xbf66b20 thru 0xbf746d4 data_size 0x7(0xdbb5 bytes) 2024-09-21T15:33:01.263 INFO:teuthology.orchestra.run.smithi107.stdout:509 trunc from 0xbf746d5 to 0xca6437e 2024-09-21T15:33:01.269 INFO:teuthology.orchestra.run.smithi107.stdout:511 punch from 0xbbcd0de to 0xbbd41e0, (0x7102 bytes) 2024-09-21T15:33:01.275 INFO:teuthology.orchestra.run.smithi107.stdout:512 trunc from 0xca6437e to 0x750277b 2024-09-21T15:33:01.279 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:01.279 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:01.279 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:01.279 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:01.279 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:01.282 INFO:teuthology.orchestra.run.smithi107.stdout:513 write 0x730d960 thru 0x731aac1 (0xd162 bytes) 2024-09-21T15:33:01.287 INFO:teuthology.orchestra.run.smithi107.stdout:514 writesame 0x4cab77f thru 0x4cac2d8 data_size 0x1(0xb5a bytes) 2024-09-21T15:33:01.292 INFO:teuthology.orchestra.run.smithi107.stdout:515 punch from 0xbfaf6 to 0xca922, (0xae2c bytes) 2024-09-21T15:33:01.295 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:01.306 INFO:teuthology.orchestra.run.smithi107.stdout:517 write 0xe1519b5 thru 0xe15c54e (0xab9a bytes) 2024-09-21T15:33:01.325 INFO:teuthology.orchestra.run.smithi107.stdout:518 trunc from 0xe15c54f to 0x1d27555 2024-09-21T15:33:01.338 INFO:teuthology.orchestra.run.smithi107.stdout:519 read 0x10942c9 thru 0x10a1b58 (0xd890 bytes) 2024-09-21T15:33:01.338 INFO:teuthology.orchestra.run.smithi107.stdout:521 read 0x61b311 thru 0x61b381 (0x71 bytes) 2024-09-21T15:33:01.340 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:01.339+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:01.356 INFO:teuthology.orchestra.run.smithi107.stdout:523 write 0x88c38a9 thru 0x88ca813 (0x6f6b bytes) 2024-09-21T15:33:01.367 INFO:teuthology.orchestra.run.smithi107.stdout:524 trunc from 0x88ca814 to 0x317a599 2024-09-21T15:33:01.398 INFO:teuthology.orchestra.run.smithi107.stdout:525 write 0x3396518 thru 0x33a1c36 (0xb71f bytes) 2024-09-21T15:33:01.404 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xe8f12dc 2024-09-21T15:33:01.415 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:01.411+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:526 trunc from 0x33a1c37 to 0xe8f12dc 2024-09-21T15:33:01.437 INFO:teuthology.orchestra.run.smithi107.stdout:527 write 0x9fee890 thru 0x9ffa5cf (0xbd40 bytes) 2024-09-21T15:33:01.445 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.459 INFO:teuthology.orchestra.run.smithi107.stdout:529 trunc from 0xe8f12dc to 0x77839ee 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.582093626999999, 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012694309000000001, 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.469 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 88 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884557+0000", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884556+0000", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884563+0000", 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5969999999999997e-06 2024-09-21T15:33:01.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884572+0000", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2599999999999994e-06 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.884582+0000", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4299999999999995e-06 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894521+0000", 2024-09-21T15:33:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099389100000000005 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.894744+0000", 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337400000000001 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.895106+0000", 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000361861 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.896480+0000", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013741000000000001 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897024+0000", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054449900000000005 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897153+0000", 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000129016 2024-09-21T15:33:01.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:30.897252+0000", 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8091999999999995e-05 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.402960306000001, 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013906523, 2024-09-21T15:33:01.475 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063691+0000", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063690+0000", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063696+0000", 2024-09-21T15:33:01.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8830000000000003e-06 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063705+0000", 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3400000000000004e-06 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063712+0000", 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1999999999999997e-06 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073726+0000", 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010013986000000001 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.073852+0000", 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125699 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.074033+0000", 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000181502 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077367+0000", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0033334409999999999 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077478+0000", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110789 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077545+0000", 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7490000000000006e-05 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077597+0000", 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.402930912999999, 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014011229, 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063720+0000", 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063719+0000", 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063725+0000", 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6069999999999998e-06 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063734+0000", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9299999999999992e-06 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.063741+0000", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9129999999999997e-06 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075299+0000", 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011557827999999999 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075415+0000", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116066 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.075607+0000", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019232099999999999 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077458+0000", 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001851339 2024-09-21T15:33:01.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077615+0000", 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015717899999999999 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077677+0000", 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1467e-05 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077731+0000", 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4196000000000001e-05 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.402383814, 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015388028, 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.487 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 115 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064267+0000", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064266+0000", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064384+0000", 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117656 2024-09-21T15:33:01.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064400+0000", 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5760000000000002e-05 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064409+0000", 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4429999999999992e-06 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076158+0000", 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011749226 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076294+0000", 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136122 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.076470+0000", 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017574600000000001 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079442+0000", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002971577 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079509+0000", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7346000000000002e-05 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079582+0000", 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3412999999999997e-05 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079655+0000", 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2649000000000002e-05 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.402230594999999, 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015786732000000001, 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.493 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 116 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064420+0000", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064419+0000", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064428+0000", 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.986e-06 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064438+0000", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8039999999999996e-06 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064445+0000", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9789999999999996e-06 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077804+0000", 2024-09-21T15:33:01.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013359207999999999 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.077921+0000", 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117039 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078131+0000", 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000209395 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079687+0000", 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001555758 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080102+0000", 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000415547 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080163+0000", 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0705999999999998e-05 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080207+0000", 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4202999999999997e-05 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.402197961999999, 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:33:01.499 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:33:01.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:33:01.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.935085620999999, 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013241467, 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 136 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:33:01.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531565+0000", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531570+0000", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0730000000000004e-06 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531579+0000", 2024-09-21T15:33:01.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1079999999999999e-06 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531585+0000", 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6730000000000003e-06 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536711+0000", 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0051255220000000004 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536808+0000", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6739999999999999e-05 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.536919+0000", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110918 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539177+0000", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002258262 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544733+0000", 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055560599999999998 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544767+0000", 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3918000000000003e-05 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544807+0000", 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9753999999999999e-05 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.935059315, 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013385154, 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 137 2024-09-21T15:33:01.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531592+0000", 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531591+0000", 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531654+0000", 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2352000000000002e-05 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531660+0000", 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1569999999999997e-06 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531663+0000", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6480000000000001e-06 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537277+0000", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056140290000000004 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537387+0000", 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109131 2024-09-21T15:33:01.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.537569+0000", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182051 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540762+0000", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031933679999999998 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544872+0000", 2024-09-21T15:33:01.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041098319999999999 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544927+0000", 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5470999999999998e-05 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.544977+0000", 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9388999999999998e-05 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.934982433999998, 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013408308000000001, 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 138 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.517 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531668+0000", 2024-09-21T15:33:01.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531671+0000", 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7659999999999999e-06 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531674+0000", 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5499999999999999e-06 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531677+0000", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0290000000000001e-06 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538356+0000", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066784920000000003 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538462+0000", 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106572 2024-09-21T15:33:01.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.538622+0000", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015995899999999999 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541174+0000", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025514589999999998 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545024+0000", 2024-09-21T15:33:01.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0038503629999999999 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545055+0000", 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0278999999999999e-05 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545077+0000", 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2177000000000001e-05 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.522 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.934970530000001, 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013559554, 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 139 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.523 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531680+0000", 2024-09-21T15:33:01.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967296 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531682+0000", 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1280000000000002e-06 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531687+0000", 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4970000000000003e-06 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531694+0000", 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4560000000000003e-06 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539257+0000", 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007562966 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539341+0000", 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4319000000000004e-05 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539437+0000", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5561000000000002e-05 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541183+0000", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017459629999999999 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545135+0000", 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0039523300000000004 2024-09-21T15:33:01.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545194+0000", 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8485e-05 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545240+0000", 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6060999999999998e-05 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.934942822, 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013608026, 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 140 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531708+0000", 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531757+0000", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9517e-05 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531765+0000", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6960000000000005e-06 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531769+0000", 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6380000000000002e-06 2024-09-21T15:33:01.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539783+0000", 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080140839999999994 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.539891+0000", 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010794900000000001 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.540060+0000", 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016991099999999999 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541649+0000", 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015888370000000001 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545266+0000", 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036169309999999999 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545294+0000", 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8170999999999998e-05 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.545316+0000", 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1642e-05 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.534 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.934877504999999, 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014580891, 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 141 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531773+0000", 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531776+0000", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7089999999999998e-06 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531779+0000", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6179999999999999e-06 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.531782+0000", 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541202+0000", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094203670000000007 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541258+0000", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6283999999999998e-05 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.541355+0000", 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6038000000000004e-05 2024-09-21T15:33:01.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.542023+0000", 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000668656 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546250+0000", 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042266659999999996 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546307+0000", 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6805999999999998e-05 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.546354+0000", 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.757e-05 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1190 3.11 3:8bc4bd9b:::rbd_data.1191bb6b225c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:01.540 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.9537250339999996, 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022028537000000001, 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1190 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:01.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512924+0000", 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512937+0000", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3223e-05 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512966+0000", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9215999999999999e-05 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512980+0000", 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4273e-05 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513106+0000", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012556000000000001 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513349+0000", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000243341 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.517115+0000", 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003766138 2024-09-21T15:33:01.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534677+0000", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017561961000000001 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534756+0000", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8382999999999998e-05 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534846+0000", 2024-09-21T15:33:01.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0075999999999996e-05 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534954+0000", 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108446 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1220 3.7 e27/21 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head v 27'13, mlcod=27'11)", 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:01.546 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.8664920839999999, 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035610309, 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600156+0000", 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615711+0000", 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015554596 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615756+0000", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5343000000000003e-05 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615775+0000", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8564e-05 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615881+0000", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010644199999999999 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615921+0000", 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9133e-05 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635654+0000", 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019733581 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635769+0000", 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0001149 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1278 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.6820977910000003, 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.060972842999999999, 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1278 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784551+0000", 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:01.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801400+0000", 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01684892 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801431+0000", 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1581999999999997e-05 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801451+0000", 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.948e-05 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801589+0000", 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013792500000000001 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801994+0000", 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040495700000000001 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.802913+0000", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00091958600000000001 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.832473+0000", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029559932000000001 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845354+0000", 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012880796999999999 2024-09-21T15:33:01.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845448+0000", 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3746000000000007e-05 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845526+0000", 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7972999999999994e-05 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1323 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.5106567139999996, 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013965239000000001, 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1323 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955994+0000", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955993+0000", 2024-09-21T15:33:01.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956001+0000", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9629999999999998e-06 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956019+0000", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8454e-05 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956031+0000", 2024-09-21T15:33:01.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2396e-05 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956102+0000", 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0840999999999995e-05 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956238+0000", 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013587399999999999 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:01.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956409+0000", 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017057300000000001 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.960773+0000", 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043641009999999996 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969864+0000", 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.009091132 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969927+0000", 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2853000000000003e-05 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969959+0000", 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2583999999999998e-05 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.562 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3973 3.e 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11281964~20966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.8985047939999999, 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.059230786000000001, 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3973 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:01.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568144+0000", 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583870+0000", 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015726602999999999 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583922+0000", 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1239000000000002e-05 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583942+0000", 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0213000000000001e-05 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584121+0000", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000179469 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584383+0000", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026151100000000002 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.585420+0000", 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0010370189999999999 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.625790+0000", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040369751000000002 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627101+0000", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001311127 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627255+0000", 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015414899999999999 2024-09-21T15:33:01.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627377+0000", 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122009 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4208 3.11 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.7571701119999998, 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054071906000000003, 2024-09-21T15:33:01.568 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4208 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709478+0000", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721564+0000", 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012085933 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721605+0000", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0862999999999998e-05 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721625+0000", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9953e-05 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721772+0000", 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014629199999999999 2024-09-21T15:33:01.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722036+0000", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000264415 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.723537+0000", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015015549999999999 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762221+0000", 2024-09-21T15:33:01.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038683842000000003 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763323+0000", 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011017150000000001 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763453+0000", 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130199 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763553+0000", 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9382000000000005e-05 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4295 3.11 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.923117+0000", 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.5435333739999999, 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014014357999999999, 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4295 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923117+0000", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923117+0000", 2024-09-21T15:33:01.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923115+0000", 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923128+0000", 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.259e-05 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923159+0000", 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0713000000000002e-05 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923174+0000", 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5406000000000001e-05 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923304+0000", 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000129756 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923587+0000", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000283761 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.924004+0000", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041639699999999999 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935917+0000", 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011913398 2024-09-21T15:33:01.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.936914+0000", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00099715799999999994 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937042+0000", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012793599999999999 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937132+0000", 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9423000000000001e-05 2024-09-21T15:33:01.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4297 3.e 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.923196+0000", 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.543454992, 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013947719000000001, 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.580 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4297 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923196+0000", 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923196+0000", 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923195+0000", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923200+0000", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8169999999999998e-06 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923210+0000", 2024-09-21T15:33:01.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0443e-05 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923218+0000", 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.413e-06 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923342+0000", 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012398899999999999 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:01.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923592+0000", 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000249855 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.924001+0000", 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040977 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935715+0000", 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011713523999999999 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.936949+0000", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001234073 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937086+0000", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013698599999999999 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937144+0000", 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7669e-05 2024-09-21T15:33:01.585 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.586 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:01.586 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:01.586 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:01.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:01.588 INFO:teuthology.orchestra.run.smithi107.stdout:531 writesame 0x7cc13c0 thru 0x7ccce9e data_size 0x1(0xbadf bytes) 2024-09-21T15:33:01.588 INFO:teuthology.orchestra.run.smithi107.stdout:532 write 0x8410e5c thru 0x841bee5 (0xb08a bytes) 2024-09-21T15:33:01.588 INFO:teuthology.orchestra.run.smithi107.stdout:533 write 0x8eccab8 thru 0x8ece63a (0x1b83 bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:534 punch from 0xb3d214 to 0xb3d462, (0x24e bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:535 write 0xa0c86ab thru 0xa0ca42f (0x1d85 bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:536 read 0x84b05a6 thru 0x84b7f09 (0x7964 bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:538 writesame 0xc5137ba thru 0xc5232c7 data_size 0x5(0xfb0e bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:539 write 0xb271210 thru 0xb27f8cb (0xe6bc bytes) 2024-09-21T15:33:01.589 INFO:teuthology.orchestra.run.smithi107.stdout:540 write 0x348dcc thru 0x3563c0 (0xd5f5 bytes) 2024-09-21T15:33:01.589 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:01.583+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:01.594 INFO:teuthology.orchestra.run.smithi107.stdout:542 punch from 0x7e259a5 to 0x7e2ebee, (0x9249 bytes) 2024-09-21T15:33:01.594 INFO:teuthology.orchestra.run.smithi107.stdout:543 read 0x44e3559 thru 0x44ea9c9 (0x7471 bytes) 2024-09-21T15:33:01.603 INFO:teuthology.orchestra.run.smithi107.stdout:544 write 0xcac7579 thru 0xcaccc33 (0x56bb bytes) 2024-09-21T15:33:01.610 INFO:teuthology.orchestra.run.smithi107.stdout:545 write 0x9231ab3 thru 0x9241829 (0xfd77 bytes) 2024-09-21T15:33:01.618 INFO:teuthology.orchestra.run.smithi107.stdout:546 read 0xb8ea612 thru 0xb8f327d (0x8c6c bytes) 2024-09-21T15:33:01.618 INFO:teuthology.orchestra.run.smithi107.stdout:547 trunc from 0xcaccc34 to 0xc85e37a 2024-09-21T15:33:01.636 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.637 INFO:teuthology.orchestra.run.smithi107.stdout:549 write 0x28486e9 thru 0x28586d3 (0xffeb bytes) 2024-09-21T15:33:01.642 INFO:teuthology.orchestra.run.smithi107.stdout:550 read 0xc212633 thru 0xc21cf41 (0xa90f bytes) 2024-09-21T15:33:01.642 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.642 INFO:teuthology.orchestra.run.smithi107.stdout:553 writesame 0x29b046c thru 0x29bde65 data_size 0x1(0xd9fa bytes) 2024-09-21T15:33:01.688 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:01.687+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:01.709 INFO:teuthology.orchestra.run.smithi107.stdout:555 trunc from 0xc85e37a to 0xe89fdd9 2024-09-21T15:33:01.720 INFO:teuthology.orchestra.run.smithi107.stdout:556 read 0xc537e6b thru 0xc54293e (0xaad4 bytes) 2024-09-21T15:33:01.720 INFO:teuthology.orchestra.run.smithi107.stdout:557 writesame 0x6fd27d3 thru 0x6fd9945 data_size 0x15(0x7173 bytes) 2024-09-21T15:33:01.723 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:01.723 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:01.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5412 3.12 3:4f6ef0b8:::rbd_object_map.11917ba4f9b:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.721041+0000", 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.0028870839999999998, 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029360860000000001, 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "waiting for sub ops", 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5412 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.724 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721041+0000", 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721041+0000", 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:01.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721039+0000", 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721051+0000", 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1199999999999999e-05 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721073+0000", 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2789000000000002e-05 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721092+0000", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8787000000000002e-05 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721169+0000", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6601999999999996e-05 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721256+0000", 2024-09-21T15:33:01.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6810000000000004e-05 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.721622+0000", 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036637600000000002 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.722544+0000", 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00092250100000000005 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723931+0000", 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00138703 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5412 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.723806+0000", 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.000122019, 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033180200000000002, 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:01.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723806+0000", 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723806+0000", 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723806+0000", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723811+0000", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8279999999999999e-06 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723823+0000", 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1119e-05 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723827+0000", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6940000000000003e-06 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723905+0000", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7657000000000005e-05 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.723921+0000", 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6419999999999999e-05 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:01.732 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 2 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:01.733 INFO:teuthology.orchestra.run.smithi107.stdout:559 trunc from 0xe89fdd9 to 0xc3fbc82 2024-09-21T15:33:01.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:01.742 INFO:teuthology.orchestra.run.smithi107.stdout:561 write 0xb55d8e4 thru 0xb560d24 (0x3441 bytes) 2024-09-21T15:33:01.761 INFO:teuthology.orchestra.run.smithi107.stdout:562 writesame 0xe7e26a0 thru 0xe7ede5e data_size 0x1(0xb7bf bytes) 2024-09-21T15:33:01.761 INFO:teuthology.orchestra.run.smithi107.stdout:563 read 0x27f9356 thru 0x28025bd (0x9268 bytes) 2024-09-21T15:33:01.762 INFO:teuthology.orchestra.run.smithi107.stdout:564 read 0x34e5fc6 thru 0x34f38c6 (0xd901 bytes) 2024-09-21T15:33:01.762 INFO:teuthology.orchestra.run.smithi107.stdout:565 read 0x1fb00f4 thru 0x1fbe930 (0xe83d bytes) 2024-09-21T15:33:01.762 INFO:teuthology.orchestra.run.smithi107.stdout:566 write 0xac0c018 thru 0xac18a9d (0xca86 bytes) 2024-09-21T15:33:01.780 INFO:teuthology.orchestra.run.smithi107.stdout:567 trunc from 0xe7ede5f to 0x8d0aa06 2024-09-21T15:33:01.788 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:01.787+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:01.799 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.799 INFO:teuthology.orchestra.run.smithi107.stdout:569 write 0x34b7b10 thru 0x34beefd (0x73ee bytes) 2024-09-21T15:33:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:571 write 0x38d16d0 thru 0x38d1ae9 (0x41a bytes) 2024-09-21T15:33:01.811 INFO:teuthology.orchestra.run.smithi107.stdout:573 read 0x327bf6d thru 0x3282a45 (0x6ad9 bytes) 2024-09-21T15:33:01.812 INFO:teuthology.orchestra.run.smithi107.stdout:575 punch from 0x487af76 to 0x4885360, (0xa3ea bytes) 2024-09-21T15:33:01.819 INFO:teuthology.orchestra.run.smithi107.stdout:576 trunc from 0x8d0aa06 to 0xbe0b829 2024-09-21T15:33:01.828 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.837 INFO:teuthology.orchestra.run.smithi107.stdout:579 writesame 0xbebc7f0 thru 0xbec0a24 data_size 0x1(0x4235 bytes) 2024-09-21T15:33:01.853 INFO:teuthology.orchestra.run.smithi107.stdout:580 trunc from 0xbec0a25 to 0x1cc4ee7 2024-09-21T15:33:01.877 INFO:teuthology.orchestra.run.smithi107.stdout:581 read 0x19ad332 thru 0x19af05f (0x1d2e bytes) 2024-09-21T15:33:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:582 read 0x5a8be6 thru 0x5b7f87 (0xf3a2 bytes) 2024-09-21T15:33:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.889 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:01.887+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:01.900 INFO:teuthology.orchestra.run.smithi107.stdout:585 write 0x8a85aa8 thru 0x8a9312a (0xd683 bytes) 2024-09-21T15:33:01.905 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:01.905 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:01.905 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:01.905 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:01.905 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:01.907 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:01.907 INFO:teuthology.orchestra.run.smithi107.stdout:587 write 0x3a5b57e thru 0x3a69b92 (0xe615 bytes) 2024-09-21T15:33:01.917 INFO:teuthology.orchestra.run.smithi107.stdout:588 write 0x8214372 thru 0x821d9f5 (0x9684 bytes) 2024-09-21T15:33:01.920 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:01.922 INFO:teuthology.orchestra.run.smithi107.stdout:589 trunc from 0x8a9312b to 0x8f84dc2 2024-09-21T15:33:01.929 INFO:teuthology.orchestra.run.smithi107.stdout:591 punch from 0x36072fb to 0x36141ad, (0xceb2 bytes) 2024-09-21T15:33:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:592 writesame 0xb10b6e4 thru 0xb110191 data_size 0x16(0x4aae bytes) 2024-09-21T15:33:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:593 punch from 0x26f4534 to 0x26ff180, (0xac4c bytes) 2024-09-21T15:33:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:594 read 0x23531c7 thru 0x235787a (0x46b4 bytes) 2024-09-21T15:33:01.940 INFO:teuthology.orchestra.run.smithi107.stdout:595 read 0x365096a thru 0x365d739 (0xcdd0 bytes) 2024-09-21T15:33:01.941 INFO:teuthology.orchestra.run.smithi107.stdout:597 trunc from 0xb110192 to 0x9bdb469 2024-09-21T15:33:01.957 INFO:teuthology.orchestra.run.smithi107.stdout:598 trunc from 0x9bdb469 to 0x4b761d3 2024-09-21T15:33:01.973 INFO:teuthology.orchestra.run.smithi107.stdout:599 writesame 0x44bf2e8 thru 0x44c483f data_size 0x4(0x5558 bytes) 2024-09-21T15:33:01.987 INFO:teuthology.orchestra.run.smithi107.stdout:600 trunc from 0x4b761d3 to 0x6d19f9a 2024-09-21T15:33:01.990 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:01.987+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:01.994 INFO:teuthology.orchestra.run.smithi107.stdout:602 write 0x53f3847 thru 0x53f8ceb (0x54a5 bytes) 2024-09-21T15:33:02.002 INFO:teuthology.orchestra.run.smithi107.stdout:603 punch from 0x5d0407e to 0x5d088eb, (0x486d bytes) 2024-09-21T15:33:02.016 INFO:teuthology.orchestra.run.smithi107.stdout:604 writesame 0xc5c9240 thru 0xc5d32be data_size 0x1(0xa07f bytes) 2024-09-21T15:33:02.016 INFO:teuthology.orchestra.run.smithi107.stdout:605 writesame 0x223ee64 thru 0x224dba0 data_size 0x1(0xed3d bytes) 2024-09-21T15:33:02.056 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:134 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'86)", 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.519241559000001, 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060115560000000004, 2024-09-21T15:33:02.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533349+0000", 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533347+0000", 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533427+0000", 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9798999999999999e-05 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533447+0000", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0207000000000002e-05 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.533461+0000", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3413e-05 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534231+0000", 2024-09-21T15:33:02.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077009699999999997 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.534286+0000", 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5556999999999997e-05 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539301+0000", 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0050144669999999999 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.539360+0000", 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9472999999999998e-05 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.517360398000001, 2024-09-21T15:33:02.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:33:02.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.067 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.516788322, 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:33:02.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:897 3.1f 3:ff84ad69:::rbd_data.11911fd826fd.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.073 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0483992890000007, 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061454999999999999, 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 897 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.074 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004190+0000", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004198+0000", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7639999999999994e-06 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004247+0000", 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8167999999999997e-05 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004260+0000", 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004323+0000", 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3482e-05 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004424+0000", 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101192 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004750+0000", 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032567099999999998 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006929+0000", 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021786119999999999 2024-09-21T15:33:02.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010188+0000", 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032598100000000001 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010282+0000", 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3350000000000006e-05 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010337+0000", 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4744999999999998e-05 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:946 3.1b 3:db55d07f:::rbd_data.11911fd826fd.000000000000001e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0374941129999993, 2024-09-21T15:33:02.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062820289999999997, 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 946 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:02.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015095+0000", 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015102+0000", 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7009999999999999e-06 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015116+0000", 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.412e-05 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0429999999999999e-05 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015450+0000", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031393000000000003 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015616+0000", 2024-09-21T15:33:02.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016547900000000001 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021378+0000", 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057623930000000002 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:981 3.1 3:812dd4d6:::rbd_data.11911fd826fd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0203050240000007, 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073781309999999996, 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 981 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032284+0000", 2024-09-21T15:33:02.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032291+0000", 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032306+0000", 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5162e-05 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032317+0000", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0641e-05 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032391+0000", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3563e-05 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032531+0000", 2024-09-21T15:33:02.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014023900000000001 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032886+0000", 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035541499999999998 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035001+0000", 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021150409999999998 2024-09-21T15:33:02.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039546+0000", 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045443339999999997 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039615+0000", 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8936000000000005e-05 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039663+0000", 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8866000000000003e-05 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:984 3.6 3:630b821a:::rbd_data.11911fd826fd.0000000000000033:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0198309519999995, 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006946069, 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-21T15:33:02.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032758+0000", 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032765+0000", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4590000000000001e-06 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032780+0000", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032789+0000", 2024-09-21T15:33:02.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4339999999999998e-06 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032836+0000", 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6891000000000002e-05 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032919+0000", 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3238000000000005e-05 2024-09-21T15:33:02.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033108+0000", 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000188803 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035121+0000", 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020131900000000002 2024-09-21T15:33:02.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044746530000000003 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039653+0000", 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6486000000000001e-05 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039705+0000", 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2935000000000001e-05 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.100 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:985 3.17 3:eca73502:::rbd_data.11911fd826fd.0000000000000034:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0194215789999994, 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0094186879999999997, 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033168+0000", 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033175+0000", 2024-09-21T15:33:02.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1729999999999998e-06 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033194+0000", 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9383999999999999e-05 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033205+0000", 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0947e-05 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033266+0000", 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0551999999999999e-05 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033373+0000", 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107155 2024-09-21T15:33:02.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033600+0000", 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022703799999999999 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035620+0000", 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020200050000000001 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042474+0000", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068545359999999996 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042537+0000", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2778000000000001e-05 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042587+0000", 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0272999999999998e-05 2024-09-21T15:33:02.108 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:982 3.10 e27/21 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0193083900000008, 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063444290000000004, 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033281+0000", 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033295+0000", 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3361e-05 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033306+0000", 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1268e-05 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033313+0000", 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6450000000000005e-06 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033365+0000", 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1208999999999997e-05 2024-09-21T15:33:02.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033391+0000", 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6353e-05 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039585+0000", 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061938990000000001 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039626+0000", 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:987 3.1f 3:fa7a7055:::rbd_data.11911fd826fd.0000000000000036:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.017601462, 2024-09-21T15:33:02.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077071170000000003, 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:02.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034988+0000", 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034996+0000", 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2360000000000004e-06 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035015+0000", 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9072e-05 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035028+0000", 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2668e-05 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035078+0000", 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0667999999999997e-05 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035162+0000", 2024-09-21T15:33:02.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3707000000000003e-05 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035389+0000", 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022708 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.037465+0000", 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020759379999999998 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042600+0000", 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051350750000000002 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042653+0000", 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3196999999999999e-05 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042696+0000", 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2800999999999999e-05 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:988 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0173951219999999, 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073861910000000003, 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035194+0000", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035203+0000", 2024-09-21T15:33:02.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2059999999999996e-06 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035222+0000", 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9320000000000001e-05 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035233+0000", 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0793e-05 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035310+0000", 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6808000000000001e-05 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035479+0000", 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016872999999999999 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:02.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035835+0000", 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035607999999999999 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039777+0000", 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039426979999999997 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042513+0000", 2024-09-21T15:33:02.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002735786 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042550+0000", 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7211e-05 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042581+0000", 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1031000000000002e-05 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3169 3.b e30/21 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head v 30'28, mlcod=29'25)", 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.167815278, 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010485235000000001, 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884773+0000", 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884789+0000", 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.556e-05 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884822+0000", 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3189999999999999e-05 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884860+0000", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8166e-05 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884989+0000", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012823500000000001 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885053+0000", 2024-09-21T15:33:02.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4503000000000002e-05 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895205+0000", 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010151415 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895260+0000", 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5736000000000003e-05 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3170 3.17 3:eb82c48b:::rbd_data.1191ab8bc3c6.0000000000000001:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.165579922, 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095184950000000001, 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3170 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887007+0000", 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887024+0000", 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.647e-05 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887072+0000", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8038999999999999e-05 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887096+0000", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3777e-05 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887229+0000", 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133805 2024-09-21T15:33:02.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887459+0000", 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229535 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887904+0000", 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044505 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892712+0000", 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048083960000000004 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896348+0000", 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036350990000000001 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896471+0000", 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012388300000000001 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896529+0000", 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7491000000000002e-05 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.139 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3172 3.b e30/21 3:d0ec678c:::rbd_data.1191ab8bc3c6.0000000000000003:head v 30'30, mlcod=29'25)", 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.1605285399999996, 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073246989999999996, 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892061+0000", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892072+0000", 2024-09-21T15:33:02.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1301000000000001e-05 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892087+0000", 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.504e-05 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892097+0000", 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969999999999993e-06 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892204+0000", 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106783 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892258+0000", 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4014000000000002e-05 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899269+0000", 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070109120000000002 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899386+0000", 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117793 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3173 3.1 3:84db22e7:::rbd_data.1191ab8bc3c6.0000000000000004:head [stat,truncate 6032748] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.1603180560000004, 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071419710000000004, 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3173 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892269+0000", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892286+0000", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6826000000000001e-05 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892323+0000", 2024-09-21T15:33:02.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7292999999999997e-05 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892345+0000", 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2186000000000001e-05 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892469+0000", 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000123566 2024-09-21T15:33:02.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892666+0000", 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019718900000000001 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893107+0000", 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044130900000000003 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895184+0000", 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020761920000000001 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899255+0000", 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040709980000000002 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899355+0000", 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100543 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899414+0000", 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9119000000000001e-05 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4150 3.1f 3:fe00fe50:::rbd_data.1191cae494be.0000000000000002:a [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.3584374129999999, 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060212410000000001, 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4150 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.152 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694155+0000", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9269999999999999e-06 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694160+0000", 2024-09-21T15:33:02.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5720000000000002e-06 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694165+0000", 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5600000000000004e-06 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.699958+0000", 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00579293 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.700050+0000", 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2114999999999995e-05 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.700174+0000", 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124459 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.34292284, 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:02.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:02.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:02.163 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.332600545, 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:02.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:02.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:02.168 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.40510331999999999, 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:02.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:02.174 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.37119188600000003, 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:02.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.179 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:02.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:02.182 INFO:teuthology.orchestra.run.smithi107.stdout:606 punch from 0x84ceda0 to 0x84d2b44, (0x3da4 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:608 read 0x6686fe3 thru 0x668ad72 (0x3d90 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:609 read 0x830d7f5 thru 0x83161b9 (0x89c5 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:610 punch from 0x589ddd5 to 0x58a111d, (0x3348 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:613 write 0xae562c thru 0xaf1f4b (0xc920 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:614 punch from 0x1debbcb to 0x1df3834, (0x7c69 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:615 write 0xc3fc3 thru 0xc5f79 (0x1fb7 bytes) 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:616 trunc from 0xc5d32bf to 0x3772764 2024-09-21T15:33:02.183 INFO:teuthology.orchestra.run.smithi107.stdout:619 read 0xc53ea3 thru 0xc56ebe (0x301c bytes) 2024-09-21T15:33:02.184 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:02.179+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:02.187 INFO:teuthology.orchestra.run.smithi107.stdout:620 write 0xabef176 thru 0xabf71f3 (0x807e bytes) 2024-09-21T15:33:02.210 INFO:teuthology.orchestra.run.smithi107.stdout:621 write 0xde656c7 thru 0xde67038 (0x1972 bytes) 2024-09-21T15:33:02.217 INFO:teuthology.orchestra.run.smithi107.stdout:622 writesame 0xc08655a thru 0xc089902 data_size 0x19(0x33a9 bytes) 2024-09-21T15:33:02.232 INFO:teuthology.orchestra.run.smithi107.stdout:623 writesame 0xeb51e34 thru 0xeb5ad2f data_size 0x2(0x8efc bytes) 2024-09-21T15:33:02.232 INFO:teuthology.orchestra.run.smithi107.stdout:624 write 0xab852c8 thru 0xab89ee4 (0x4c1d bytes) 2024-09-21T15:33:02.244 INFO:teuthology.orchestra.run.smithi107.stdout:625 trunc from 0xeb5ad30 to 0xa9ace7a 2024-09-21T15:33:02.262 INFO:teuthology.orchestra.run.smithi107.stdout:627 read 0x952c118 thru 0x952ec7a (0x2b63 bytes) 2024-09-21T15:33:02.262 INFO:teuthology.orchestra.run.smithi107.stdout:628 write 0x25dca09 thru 0x25e9c9e (0xd296 bytes) 2024-09-21T15:33:02.268 INFO:teuthology.orchestra.run.smithi107.stdout:629 write 0x71ddc6d thru 0x71e6583 (0x8917 bytes) 2024-09-21T15:33:02.282 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:02.279+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:02.290 INFO:teuthology.orchestra.run.smithi107.stdout:630 trunc from 0xa9ace7a to 0x42bb73c 2024-09-21T15:33:02.317 INFO:teuthology.orchestra.run.smithi107.stdout:631 trunc from 0x42bb73c to 0x6ff354f 2024-09-21T15:33:02.323 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5686 3.6 e33/21 3:65672256:::rbd_header.11917ba4f9b:head v 33'63, mlcod=33'61)", 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.322787+0000", 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.0013014809999999999, 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013188799999999999, 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322787+0000", 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322787+0000", 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322785+0000", 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322797+0000", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2126e-05 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322818+0000", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1265000000000001e-05 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322828+0000", 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0025e-05 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.322942+0000", 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114041 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.323006+0000", 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4005999999999997e-05 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.327 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:02.328 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 1 2024-09-21T15:33:02.328 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:02.330 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:02.330 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:02.330 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:33:02.330 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.000000 2024-09-21T15:33:02.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0 2024-09-21T15:33:02.342 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:02.346 INFO:teuthology.orchestra.run.smithi107.stdout:634 writesame 0xa8ebc19 thru 0xa8ec1dd data_size 0x1(0x5c5 bytes) 2024-09-21T15:33:02.346 INFO:teuthology.orchestra.run.smithi107.stdout:635 write 0xa0b38f0 thru 0xa0c2749 (0xee5a bytes) 2024-09-21T15:33:02.352 INFO:teuthology.orchestra.run.smithi107.stdout:636 read 0x87b7c4b thru 0x87c038e (0x8744 bytes) 2024-09-21T15:33:02.363 INFO:teuthology.orchestra.run.smithi107.stdout:637 write 0xc98c2df thru 0xc99ac29 (0xe94b bytes) 2024-09-21T15:33:02.369 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:02.369 INFO:teuthology.orchestra.run.smithi107.stdout:640 writesame 0x797291b thru 0x7976c68 data_size 0x5(0x434e bytes) 2024-09-21T15:33:02.376 INFO:teuthology.orchestra.run.smithi107.stdout:641 punch from 0x6b383f4 to 0x6b3b541, (0x314d bytes) 2024-09-21T15:33:02.376 INFO:teuthology.orchestra.run.smithi107.stdout:642 read 0x3d618c2 thru 0x3d6ad01 (0x9440 bytes) 2024-09-21T15:33:02.376 INFO:teuthology.orchestra.run.smithi107.stdout:644 read 0xad0f466 thru 0xad18ff4 (0x9b8f bytes) 2024-09-21T15:33:02.382 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:02.379+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:02.390 INFO:teuthology.orchestra.run.smithi107.stdout:645 trunc from 0xc99ac2a to 0x4113e56 2024-09-21T15:33:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:646 write 0x9229542 thru 0x922e292 (0x4d51 bytes) 2024-09-21T15:33:02.429 INFO:teuthology.orchestra.run.smithi107.stdout:647 clone 10 order 23 su 32768 sc 4 2024-09-21T15:33:02.468 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:02.468 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:02.468 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:02.468 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:02.468 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:02.483 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:02.479+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:02.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:02.584 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:02.583+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.645 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.911392165000001, 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:33:02.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:33:02.649 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:621 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 557257~27160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.219563618, 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031532890000000001, 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.650 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 621 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423631+0000", 2024-09-21T15:33:02.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423627+0000", 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423663+0000", 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5459000000000001e-05 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423699+0000", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6826000000000003e-05 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423716+0000", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6654e-05 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.423816+0000", 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969000000000005e-05 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424070+0000", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025371399999999999 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424403+0000", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033311799999999998 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426114+0000", 2024-09-21T15:33:02.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001711171 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426619+0000", 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00050520199999999995 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426719+0000", 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9313000000000003e-05 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426784+0000", 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5629000000000005e-05 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:641 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write 490537~26655] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.169475050999999, 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003364204, 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 641 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473716+0000", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473753+0000", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6430000000000002e-05 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473790+0000", 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:33:02.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473827+0000", 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6779999999999997e-05 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473923+0000", 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6143000000000004e-05 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474178+0000", 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000255374 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474426+0000", 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024796799999999999 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.475922+0000", 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001495515 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476887+0000", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00096475599999999999 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476972+0000", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5562999999999994e-05 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.477084+0000", 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111319 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.661 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.100980543, 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:33:02.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:02.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:688 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.09134796, 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032222280000000002, 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 688 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551845+0000", 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551852+0000", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6499999999999999e-06 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551870+0000", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7853e-05 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551882+0000", 2024-09-21T15:33:02.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2459999999999999e-05 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554681+0000", 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002798189 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554810+0000", 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012915499999999999 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.555069+0000", 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025897799999999999 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:716 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.065424740999999, 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034336240000000001, 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 716 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577768+0000", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577777+0000", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4470000000000006e-06 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577796+0000", 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9769999999999999e-05 2024-09-21T15:33:02.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577811+0000", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4686e-05 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580639+0000", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028279590000000001 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580881+0000", 2024-09-21T15:33:02.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241955 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581203+0000", 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000322267 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.065369393999999, 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:33:02.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.680 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.065288959, 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:02.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.684 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.065243815000001, 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.685 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.065065586999999, 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:02.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:02.693 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.064991937, 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:33:02.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:33:02.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6469851099999993, 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:33:02.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 25 entries 27'10 (0'0) error 3:5afb9d3f:::rbd_data.11911fd826fd.000000000000000c:head by client.4497.0:884 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 26'7)", 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6449428289999997, 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030724049999999998, 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998252+0000", 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998251+0000", 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998264+0000", 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3871e-05 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998288+0000", 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3693000000000002e-05 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998307+0000", 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8612000000000001e-05 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.998382+0000", 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5172999999999993e-05 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.001324+0000", 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029421310000000002 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.1a epoch 27/21 rep_tid 27 entries 27'12 (0'0) error 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head by client.4497.0:904 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 27'9)", 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6362899710000001, 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003020773, 2024-09-21T15:33:02.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006904+0000", 2024-09-21T15:33:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006903+0000", 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006916+0000", 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2512e-05 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006939+0000", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3274000000000001e-05 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006955+0000", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5383000000000002e-05 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.007055+0000", 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100079 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.009925+0000", 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028706690000000002 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:943 3.1a e27/21 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head v 27'13, mlcod=27'11)", 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6280918110000009, 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034511770000000002, 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.711 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015101+0000", 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015110+0000", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6519999999999995e-06 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015127+0000", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7030000000000001e-05 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8440000000000004e-06 2024-09-21T15:33:02.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015199+0000", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2786999999999994e-05 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015229+0000", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9927e-05 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018501+0000", 2024-09-21T15:33:02.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032721299999999998 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018554+0000", 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2970999999999998e-05 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.8 epoch 27/21 rep_tid 41 entries 27'3 (0'0) error 3:15312f39:::rbd_data.11911fd826fd.0000000000000020:head by client.4497.0:950 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6259955609999999, 2024-09-21T15:33:02.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037814699999999999, 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017205+0000", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7449999999999998e-06 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017223+0000", 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7626999999999999e-05 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017241+0000", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.749e-05 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017302+0000", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1802000000000002e-05 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.020980+0000", 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036779909999999998 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.718 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:988 3.6 e27/21 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head v 27'10, mlcod=26'6)", 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.6072384520000007, 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036395920000000001, 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035955+0000", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035964+0000", 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:33:02.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035980+0000", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5855e-05 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035988+0000", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8650000000000001e-06 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036064+0000", 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6191000000000003e-05 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036088+0000", 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3470000000000001e-05 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039519+0000", 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034318809999999999 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6165e-05 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5390 3.6 e33/21 3:65672256:::rbd_header.11917ba4f9b:head v 33'40, mlcod=33'38)", 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.015476125, 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032420679999999999, 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.723 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627716+0000", 2024-09-21T15:33:02.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627732+0000", 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.571e-05 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627765+0000", 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3210000000000002e-05 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627779+0000", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3993e-05 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627850+0000", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1402000000000005e-05 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627884+0000", 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3318000000000002e-05 2024-09-21T15:33:02.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.630912+0000", 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030284999999999999 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.630960+0000", 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8267999999999999e-05 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5584 3.1a e33/21 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head v 33'40, mlcod=32'38)", 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.59098503199999997, 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053598999999999999, 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052208+0000", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056205+0000", 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003997202 2024-09-21T15:33:02.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056221+0000", 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6167e-05 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056231+0000", 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.55e-06 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056297+0000", 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6231999999999999e-05 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056329+0000", 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1977999999999998e-05 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057476+0000", 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011471280000000001 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057569+0000", 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3178999999999997e-05 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5597 3.8 e33/21 3:16129f72:::rbd_data.11917ba4f9b.0000000000000000:head v 33'15, mlcod=33'13)", 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.108389+0000", 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.53480577399999996, 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029992959999999998, 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:02.732 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108389+0000", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108389+0000", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108387+0000", 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:02.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108786+0000", 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039875700000000002 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108803+0000", 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.738e-05 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:02.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108811+0000", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7579999999999992e-06 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108893+0000", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1389000000000003e-05 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.108916+0000", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3339000000000001e-05 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:02.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.111361+0000", 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024446329999999999 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.111388+0000", 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7438999999999999e-05 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:02.736 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:02.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:02.738 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:02.735+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:02.838 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:02.835+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:02.911 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:02.911 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:02.911 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:02.911 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:02.913 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.3 primary-affinity to 0 (802) 2024-09-21T15:33:02.921 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:33:02.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:02.944 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:02.939+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:03.039 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:03.035+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:03.107 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:03.108 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:03.108 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:03.108 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:03.108 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:03.122 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:03.140 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:03.139+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:03.241 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:03.239+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:03.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:03.293 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:03.293 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:03.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:03.293 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:621 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'7, mlcod=25'5)", 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.424612+0000", 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.867448622, 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001731521, 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424612+0000", 2024-09-21T15:33:03.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424612+0000", 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424609+0000", 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424678+0000", 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8343000000000006e-05 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424710+0000", 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2356e-05 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424725+0000", 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4833e-05 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424838+0000", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113096 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.424898+0000", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0372999999999999e-05 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426255+0000", 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00135622 2024-09-21T15:33:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.426344+0000", 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8882999999999996e-05 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:623 3.9 e26/21 3:920e09a4:::rbd_data.11911fd826fd.00000000000000ce:head v 26'3, mlcod=26'1)", 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.430603+0000", 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.861458137, 2024-09-21T15:33:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001685733, 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430603+0000", 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430603+0000", 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430600+0000", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430618+0000", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7350000000000002e-05 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430644+0000", 2024-09-21T15:33:03.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6112999999999999e-05 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430657+0000", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2744e-05 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430777+0000", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119849 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.430817+0000", 2024-09-21T15:33:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0420000000000003e-05 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.432177+0000", 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001360289 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.432288+0000", 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110982 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:627 3.9 e26/21 3:920e09a4:::rbd_data.11911fd826fd.00000000000000ce:head v 26'4, mlcod=26'2)", 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.445731+0000", 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.846330064, 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001485985, 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445731+0000", 2024-09-21T15:33:03.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445731+0000", 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445728+0000", 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445783+0000", 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4518999999999998e-05 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445825+0000", 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2410000000000002e-05 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445852+0000", 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6726000000000001e-05 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.445973+0000", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012176300000000001 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.446035+0000", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1266000000000006e-05 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.447125+0000", 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010903149999999999 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.447217+0000", 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1578999999999999e-05 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:641 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'8, mlcod=26'6)", 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.817401437999999, 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001951022, 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474657+0000", 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474747+0000", 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9559000000000001e-05 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474777+0000", 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.991e-05 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474790+0000", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3246e-05 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474904+0000", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011387600000000001 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474973+0000", 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8847e-05 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476526+0000", 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015536180000000001 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476610+0000", 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3931999999999995e-05 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.311 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.748119102, 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:33:03.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:33:03.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:677 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'10, mlcod=26'7)", 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.546355+0000", 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.745705491000001, 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001645741, 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546355+0000", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546355+0000", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546353+0000", 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546366+0000", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2697000000000001e-05 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546391+0000", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4386e-05 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546410+0000", 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9752999999999999e-05 2024-09-21T15:33:03.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546486+0000", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5430000000000001e-05 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.546520+0000", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4168999999999998e-05 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547941+0000", 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0014207289999999999 2024-09-21T15:33:03.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.548001+0000", 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0257000000000003e-05 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:738 3.9 e26/21 3:9661011e:::rbd_data.11911fd826fd.0000000000000032:head v 26'5, mlcod=26'3)", 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.704571785000001, 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017551730000000001, 2024-09-21T15:33:03.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587487+0000", 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587502+0000", 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5460000000000001e-05 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587531+0000", 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.836e-05 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587551+0000", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0299999999999999e-05 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587650+0000", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9412000000000003e-05 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587709+0000", 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8536000000000003e-05 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.589182+0000", 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001473495 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.589244+0000", 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1729999999999993e-05 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:875 3.4 e27/21 3:21d8c9f2:::rbd_data.11911fd826fd.0000000000000006:head v 27'3, mlcod=24'1)", 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.994396+0000", 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.297665110000001, 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001712246, 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994396+0000", 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994396+0000", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994394+0000", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994406+0000", 2024-09-21T15:33:03.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2300000000000001e-05 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994449+0000", 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2305000000000002e-05 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994459+0000", 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0706e-05 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994513+0000", 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3439000000000001e-05 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.994541+0000", 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8653000000000002e-05 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996062+0000", 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001520347 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996108+0000", 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6056000000000001e-05 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:958 3.9 e27/21 3:91233dae:::rbd_data.11911fd826fd.0000000000000021:head v 27'9, mlcod=27'7)", 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.021511+0000", 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.270549508, 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001466644, 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021511+0000", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021511+0000", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021510+0000", 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021522+0000", 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2033e-05 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021538+0000", 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5860000000000001e-05 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021545+0000", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8900000000000001e-06 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021592+0000", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7258000000000001e-05 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021615+0000", 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3217000000000001e-05 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.022929+0000", 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013141069999999999 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.022978+0000", 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8488000000000001e-05 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3130 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'17, mlcod=28'14)", 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.4683839839999999, 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018694930000000001, 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823676+0000", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823685+0000", 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823711+0000", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6106e-05 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823726+0000", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4644000000000001e-05 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823829+0000", 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102505 2024-09-21T15:33:03.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823867+0000", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8606999999999998e-05 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825433+0000", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015654810000000001 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825546+0000", 2024-09-21T15:33:03.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113416 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.2772028310000003, 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:33:03.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:03.342 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.2674127390000001, 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:33:03.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:33:03.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.2534586990000003, 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:33:03.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.2452915539999996, 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:03.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:33:03.355 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.244545155, 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:33:03.359 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.2261541789999999, 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:33:03.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.1652882389999997, 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:33:03.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:33:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 6.1536179029999998, 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3524 3.9 e31/21 3:91379144:::rbd_object_map.1191cae494be:head v 31'32, mlcod=30'30)", 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.897361+0000", 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.3946997310000002, 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011191829999999999, 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897361+0000", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897361+0000", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897361+0000", 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897368+0000", 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2679999999999997e-06 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897391+0000", 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2716000000000001e-05 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897401+0000", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0456e-05 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897478+0000", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7154e-05 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.897509+0000", 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0830000000000001e-05 2024-09-21T15:33:03.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.898435+0000", 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00092640799999999998 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.898480+0000", 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4662e-05 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.377 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3986 3.9 e31/21 3:931f7677:::rbd_data.119117b26f37.0000000000000007:head v 31'34, mlcod=31'32)", 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.637422+0000", 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.6546384779999999, 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015123879999999999, 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637422+0000", 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637422+0000", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637420+0000", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637860+0000", 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044040799999999999 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637893+0000", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2443000000000001e-05 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637907+0000", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4337e-05 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637972+0000", 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5314999999999994e-05 2024-09-21T15:33:03.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.637998+0000", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6016999999999998e-05 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.638896+0000", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00089785199999999998 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.638935+0000", 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8254000000000003e-05 2024-09-21T15:33:03.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:03.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:03.382 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:03.383 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:03.384 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:03.379+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:03.410 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:03.410 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:03.426 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:03.427 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:03.443 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:03.443 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:03.459 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:03.480 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:03.480 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:03.484 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:03.483+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:03.500 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:03.500 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:03.521 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:03.521 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:03.525 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:03.525 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:03.539 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:03.542 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:03.584 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:03.583+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:03.660 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:03.661 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:03.675 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:03.685 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:03.683+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:03.786 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:03.783+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:03.797 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:03.797 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:03.813 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:03.887 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:03.887+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:03.915 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:33:03.953 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:03.953 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:03.969 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:03.982 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone9 from 0x922e293 (overlap 0x1ffac4) to 0x1662bc 2024-09-21T15:33:04.011 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:04.011+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:04.089 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:04.087+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:04.090 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T15:33:04.134 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.134 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.150 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:04.191 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:04.191+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:04.281 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.290 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:04.287+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:04.305 INFO:teuthology.orchestra.run.smithi107.stdout:650 write 0xcc72273 thru 0xcc793f9 (0x7187 bytes) 2024-09-21T15:33:04.316 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.316 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.330 INFO:teuthology.orchestra.run.smithi107.stdout:652 writesame 0xe884266 thru 0xe89196e data_size 0x1(0xd709 bytes) 2024-09-21T15:33:04.330 INFO:teuthology.orchestra.run.smithi107.stdout:653 punch from 0xe71cab to 0xe7524a, (0x359f bytes) 2024-09-21T15:33:04.330 INFO:teuthology.orchestra.run.smithi107.stdout:654 write 0x1f9f2ed thru 0x1fae323 (0xf037 bytes) 2024-09-21T15:33:04.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:04.362 INFO:teuthology.orchestra.run.smithi107.stdout:655 read 0xc2617fe thru 0xc2647fa (0x2ffd bytes) 2024-09-21T15:33:04.362 INFO:teuthology.orchestra.run.smithi107.stdout:656 read 0x5e114be thru 0x5e1d711 (0xc254 bytes) 2024-09-21T15:33:04.362 INFO:teuthology.orchestra.run.smithi107.stdout:657 write 0xe30290d thru 0xe3090ae (0x67a2 bytes) 2024-09-21T15:33:04.370 INFO:teuthology.orchestra.run.smithi107.stdout:660 punch from 0x45947f6 to 0x459c2fa, (0x7b04 bytes) 2024-09-21T15:33:04.370 INFO:teuthology.orchestra.run.smithi107.stdout:662 punch from 0xe1c58f7 to 0xe1d32f3, (0xd9fc bytes) 2024-09-21T15:33:04.391 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:04.387+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:04.396 INFO:teuthology.orchestra.run.smithi107.stdout:663 trunc from 0xe89196f to 0x5103849 2024-09-21T15:33:04.427 INFO:teuthology.orchestra.run.smithi107.stdout:665 punch from 0x1e4ef41 to 0x1e56ffa, (0x80b9 bytes) 2024-09-21T15:33:04.427 INFO:teuthology.orchestra.run.smithi107.stdout:666 read 0x479aa64 thru 0x47a3a4a (0x8fe7 bytes) 2024-09-21T15:33:04.427 INFO:teuthology.orchestra.run.smithi107.stdout:668 read 0x3327da6 thru 0x332ea01 (0x6c5c bytes) 2024-09-21T15:33:04.428 INFO:teuthology.orchestra.run.smithi107.stdout:669 read 0x324d51 thru 0x32ddc6 (0x9076 bytes) 2024-09-21T15:33:04.428 INFO:teuthology.orchestra.run.smithi107.stdout:670 punch from 0x4eba97d to 0x4ec4392, (0x9a15 bytes) 2024-09-21T15:33:04.440 INFO:teuthology.orchestra.run.smithi107.stdout:671 write 0x549229f thru 0x5493c23 (0x1985 bytes) 2024-09-21T15:33:04.443 INFO:teuthology.orchestra.run.smithi107.stdout:672 write 0x21527e8 thru 0x215b83a (0x9053 bytes) 2024-09-21T15:33:04.462 INFO:teuthology.orchestra.run.smithi107.stdout:673 write 0x27dde6d thru 0x27df5ab (0x173f bytes) 2024-09-21T15:33:04.463 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.463 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.478 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:04.480 INFO:teuthology.orchestra.run.smithi107.stdout:674 write 0x6f5c7e1 thru 0x6f62383 (0x5ba3 bytes) 2024-09-21T15:33:04.489 INFO:teuthology.orchestra.run.smithi107.stdout:675 writesame 0x2afceb0 thru 0x2afdf44 data_size 0x5(0x1095 bytes) 2024-09-21T15:33:04.511 INFO:teuthology.orchestra.run.smithi107.stdout:676 writesame 0xa954fc6 thru 0xa963a1e data_size 0x1(0xea59 bytes) 2024-09-21T15:33:04.511 INFO:teuthology.orchestra.run.smithi107.stdout:679 punch from 0x7b1bbea to 0x7b250fd, (0x9513 bytes) 2024-09-21T15:33:04.511 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.511 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.511 INFO:teuthology.orchestra.run.smithi107.stdout:683 write 0x65fc622 thru 0x6606e4f (0xa82e bytes) 2024-09-21T15:33:04.513 INFO:teuthology.orchestra.run.smithi107.stdout:684 write 0x1c988f9 thru 0x1c9db63 (0x526b bytes) 2024-09-21T15:33:04.515 INFO:teuthology.orchestra.run.smithi107.stdout:685 read 0x4752902 thru 0x475c11d (0x981c bytes) 2024-09-21T15:33:04.516 INFO:teuthology.orchestra.run.smithi107.stdout:686 trunc from 0xa963a1f to 0xab5b464 2024-09-21T15:33:04.523 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:04.523+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:04.525 INFO:teuthology.orchestra.run.smithi107.stdout:687 write 0x9fd42bb thru 0x9fdf44b (0xb191 bytes) 2024-09-21T15:33:04.534 INFO:teuthology.orchestra.run.smithi107.stdout:688 punch from 0x63afd63 to 0x63b3398, (0x3635 bytes) 2024-09-21T15:33:04.534 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.552 INFO:teuthology.orchestra.run.smithi107.stdout:690 writesame 0xe500a2e thru 0xe503300 data_size 0x1(0x28d3 bytes) 2024-09-21T15:33:04.552 INFO:teuthology.orchestra.run.smithi107.stdout:691 punch from 0xd012fec to 0xd01c0d6, (0x90ea bytes) 2024-09-21T15:33:04.552 INFO:teuthology.orchestra.run.smithi107.stdout:692 punch from 0x235088d to 0x235ec21, (0xe394 bytes) 2024-09-21T15:33:04.552 INFO:teuthology.orchestra.run.smithi107.stdout:693 write 0x4d06684 thru 0x4d1410c (0xda89 bytes) 2024-09-21T15:33:04.555 INFO:teuthology.orchestra.run.smithi107.stdout:694 read 0xc34bc8d thru 0xc351b12 (0x5e86 bytes) 2024-09-21T15:33:04.555 INFO:teuthology.orchestra.run.smithi107.stdout:697 read 0xb11b898 thru 0xb12190e (0x6077 bytes) 2024-09-21T15:33:04.555 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.555 INFO:teuthology.orchestra.run.smithi107.stdout:700 write 0x4e5d2ae thru 0x4e65142 (0x7e95 bytes) 2024-09-21T15:33:04.558 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.558 INFO:teuthology.orchestra.run.smithi107.stdout:702 write 0x76fdf2a thru 0x76fe6ac (0x783 bytes) 2024-09-21T15:33:04.560 INFO:teuthology.orchestra.run.smithi107.stdout:703 writesame 0x6d96823 thru 0x6d97ca1 data_size 0x21(0x147f bytes) 2024-09-21T15:33:04.580 INFO:teuthology.orchestra.run.smithi107.stdout:704 trunc from 0xe503301 to 0x8c5e6f9 2024-09-21T15:33:04.592 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:04.587+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:04.592 INFO:teuthology.orchestra.run.smithi107.stdout:705 read 0x8a000a9 thru 0x8a0c572 (0xc4ca bytes) 2024-09-21T15:33:04.592 INFO:teuthology.orchestra.run.smithi107.stdout:707 read 0x1a7b7b3 thru 0x1a83efd (0x874b bytes) 2024-09-21T15:33:04.594 INFO:teuthology.orchestra.run.smithi107.stdout:708 trunc from 0x8c5e6f9 to 0x8409663 2024-09-21T15:33:04.599 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.600 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.614 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:04.622 INFO:teuthology.orchestra.run.smithi107.stdout:711 trunc from 0x8409663 to 0xd3ea431 2024-09-21T15:33:04.656 INFO:teuthology.orchestra.run.smithi107.stdout:712 trunc from 0xd3ea431 to 0x31eb114 2024-09-21T15:33:04.694 INFO:teuthology.orchestra.run.smithi107.stdout:714 read 0xf9bd42 thru 0xfa66b7 (0xa976 bytes) 2024-09-21T15:33:04.699 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:04.699+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:04.706 INFO:teuthology.orchestra.run.smithi107.stdout:715 write 0x41898b0 thru 0x41909bc (0x710d bytes) 2024-09-21T15:33:04.720 INFO:teuthology.orchestra.run.smithi107.stdout:716 trunc from 0x41909bd to 0x646713c 2024-09-21T15:33:04.748 INFO:teuthology.orchestra.run.smithi107.stdout:717 trunc from 0x646713c to 0xd7e743e 2024-09-21T15:33:04.760 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.760 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.760 INFO:teuthology.orchestra.run.smithi107.stdout:720 writesame 0xbd4a975 thru 0xbd51ee5 data_size 0x7(0x7571 bytes) 2024-09-21T15:33:04.770 INFO:teuthology.orchestra.run.smithi107.stdout:722 punch from 0xc0b2ca5 to 0xc0ba4f4, (0x784f bytes) 2024-09-21T15:33:04.770 INFO:teuthology.orchestra.run.smithi107.stdout:723 write 0x480110a thru 0x480e522 (0xd419 bytes) 2024-09-21T15:33:04.776 INFO:teuthology.orchestra.run.smithi107.stdout:725 write 0x23b76b6 thru 0x23bf961 (0x82ac bytes) 2024-09-21T15:33:04.779 INFO:teuthology.orchestra.run.smithi107.stdout:726 write 0x7d3d1ae thru 0x7d400d7 (0x2f2a bytes) 2024-09-21T15:33:04.783 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.783 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.787 INFO:teuthology.orchestra.run.smithi107.stdout:727 write 0x2285cc1 thru 0x228b476 (0x57b6 bytes) 2024-09-21T15:33:04.790 INFO:teuthology.orchestra.run.smithi107.stdout:729 write 0xd43576f thru 0xd437533 (0x1dc5 bytes) 2024-09-21T15:33:04.793 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:04.791+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:04.797 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:04.811 INFO:teuthology.orchestra.run.smithi107.stdout:731 trunc from 0xd7e743e to 0x86c06f8 2024-09-21T15:33:04.832 INFO:teuthology.orchestra.run.smithi107.stdout:732 read 0x4221df9 thru 0x422f623 (0xd82b bytes) 2024-09-21T15:33:04.833 INFO:teuthology.orchestra.run.smithi107.stdout:734 read 0x224fba0 thru 0x225f63e (0xfa9f bytes) 2024-09-21T15:33:04.835 INFO:teuthology.orchestra.run.smithi107.stdout:735 punch from 0x7954f48 to 0x79592ef, (0x43a7 bytes) 2024-09-21T15:33:04.835 INFO:teuthology.orchestra.run.smithi107.stdout:736 write 0x3d1df53 thru 0x3d23af4 (0x5ba2 bytes) 2024-09-21T15:33:04.838 INFO:teuthology.orchestra.run.smithi107.stdout:737 read 0x49b6a60 thru 0x49c16a0 (0xac41 bytes) 2024-09-21T15:33:04.850 INFO:teuthology.orchestra.run.smithi107.stdout:740 trunc from 0x86c06f8 to 0xca58698 2024-09-21T15:33:04.860 INFO:teuthology.orchestra.run.smithi107.stdout:741 read 0x3588b69 thru 0x358fdc8 (0x7260 bytes) 2024-09-21T15:33:04.861 INFO:teuthology.orchestra.run.smithi107.stdout:742 punch from 0xb36c402 to 0xb371809, (0x5407 bytes) 2024-09-21T15:33:04.861 INFO:teuthology.orchestra.run.smithi107.stdout:743 read 0x7654e87 thru 0x765ab21 (0x5c9b bytes) 2024-09-21T15:33:04.862 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.862 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:04.883 INFO:teuthology.orchestra.run.smithi107.stdout:747 trunc from 0xca58698 to 0x54681ac 2024-09-21T15:33:04.894 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:04.891+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:04.904 INFO:teuthology.orchestra.run.smithi107.stdout:748 write 0x56ddf6 thru 0x5730d0 (0x52db bytes) 2024-09-21T15:33:04.923 INFO:teuthology.orchestra.run.smithi107.stdout:749 read 0x4dfa8f6 thru 0x4e042e4 (0x99ef bytes) 2024-09-21T15:33:04.949 INFO:teuthology.orchestra.run.smithi107.stdout:750 write 0x9fec608 thru 0x9ff8554 (0xbf4d bytes) 2024-09-21T15:33:04.961 INFO:teuthology.orchestra.run.smithi107.stdout:751 write 0x1f4eccd thru 0x1f54705 (0x5a39 bytes) 2024-09-21T15:33:04.978 INFO:teuthology.orchestra.run.smithi107.stdout:753 write 0xb56a5a4 thru 0xb572afc (0x8559 bytes) 2024-09-21T15:33:04.979 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:04.979 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:04.994 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:04.995 INFO:teuthology.orchestra.run.smithi107.stdout:754 trunc from 0xb572afd to 0xe678829 2024-09-21T15:33:04.995 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:04.995+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:05.005 INFO:teuthology.orchestra.run.smithi107.stdout:755 read 0x1947368 thru 0x19483e7 (0x1080 bytes) 2024-09-21T15:33:05.006 INFO:teuthology.orchestra.run.smithi107.stdout:756 read 0xa2e255c thru 0xa2e5faf (0x3a54 bytes) 2024-09-21T15:33:05.007 INFO:teuthology.orchestra.run.smithi107.stdout:758 writesame 0x40a4b90 thru 0x40ab169 data_size 0x1(0x65da bytes) 2024-09-21T15:33:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:761 read 0xd0c2476 thru 0xd0c5f5f (0x3aea bytes) 2024-09-21T15:33:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:762 read 0x3a8c90f thru 0x3a91b31 (0x5223 bytes) 2024-09-21T15:33:05.017 INFO:teuthology.orchestra.run.smithi107.stdout:763 writesame 0x56d8d0f thru 0x56e4227 data_size 0x25(0xb519 bytes) 2024-09-21T15:33:05.025 INFO:teuthology.orchestra.run.smithi107.stdout:764 punch from 0x6958f45 to 0x6960cdd, (0x7d98 bytes) 2024-09-21T15:33:05.025 INFO:teuthology.orchestra.run.smithi107.stdout:765 read 0x99066b9 thru 0x9909437 (0x2d7f bytes) 2024-09-21T15:33:05.025 INFO:teuthology.orchestra.run.smithi107.stdout:767 read 0xc28db9d thru 0xc29a907 (0xcd6b bytes) 2024-09-21T15:33:05.026 INFO:teuthology.orchestra.run.smithi107.stdout:768 write 0x8a4b49f thru 0x8a56f09 (0xba6b bytes) 2024-09-21T15:33:05.029 INFO:teuthology.orchestra.run.smithi107.stdout:770 read 0xab21bf2 thru 0xab2d5d8 (0xb9e7 bytes) 2024-09-21T15:33:05.029 INFO:teuthology.orchestra.run.smithi107.stdout:771 read 0xdd585a9 thru 0xdd5a28b (0x1ce3 bytes) 2024-09-21T15:33:05.030 INFO:teuthology.orchestra.run.smithi107.stdout:772 read 0x331f801 thru 0x33272c3 (0x7ac3 bytes) 2024-09-21T15:33:05.030 INFO:teuthology.orchestra.run.smithi107.stdout:776 writesame 0xd6b4edd thru 0xd6bda16 data_size 0x47(0x8b3a bytes) 2024-09-21T15:33:05.038 INFO:teuthology.orchestra.run.smithi107.stdout:778 read 0xa921b20 thru 0xa924ff7 (0x34d8 bytes) 2024-09-21T15:33:05.047 INFO:teuthology.orchestra.run.smithi107.stdout:779 write 0xeb3065f thru 0xeb3b484 (0xae26 bytes) 2024-09-21T15:33:05.054 INFO:teuthology.orchestra.run.smithi107.stdout:782 read 0xc0b0675 thru 0xc0b9fd5 (0x9961 bytes) 2024-09-21T15:33:05.054 INFO:teuthology.orchestra.run.smithi107.stdout:784 read 0xd5937f4 thru 0xd5947f7 (0x1004 bytes) 2024-09-21T15:33:05.055 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.055 INFO:teuthology.orchestra.run.smithi107.stdout:786 writesame 0x95b51e0 thru 0x95c49c8 data_size 0x5(0xf7e9 bytes) 2024-09-21T15:33:05.071 INFO:teuthology.orchestra.run.smithi107.stdout:787 write 0xd0d1bb9 thru 0xd0de6c0 (0xcb08 bytes) 2024-09-21T15:33:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:789 punch from 0x3ccba1a to 0x3cd5aa0, (0xa086 bytes) 2024-09-21T15:33:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:793 punch from 0x13ee55c to 0x13fbd4f, (0xd7f3 bytes) 2024-09-21T15:33:05.075 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.096 INFO:teuthology.orchestra.run.smithi107.stdout:795 trunc from 0xeb3b485 to 0x6ec56de 2024-09-21T15:33:05.097 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:05.095+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:05.125 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:05.125 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:05.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:05.145 INFO:teuthology.orchestra.run.smithi107.stdout:797 write 0xc366a1b thru 0xc36f9dd (0x8fc3 bytes) 2024-09-21T15:33:05.153 INFO:teuthology.orchestra.run.smithi107.stdout:798 write 0x350918e thru 0x350b3c3 (0x2236 bytes) 2024-09-21T15:33:05.156 INFO:teuthology.orchestra.run.smithi107.stdout:799 write 0xa505 thru 0x15682 (0xb17e bytes) 2024-09-21T15:33:05.159 INFO:teuthology.orchestra.run.smithi107.stdout:800 write 0x9202a9e thru 0x92053a3 (0x2906 bytes) 2024-09-21T15:33:05.164 INFO:teuthology.orchestra.run.smithi107.stdout:804 write 0xa949819 thru 0xa94a754 (0xf3c bytes) 2024-09-21T15:33:05.172 INFO:teuthology.orchestra.run.smithi107.stdout:805 punch from 0x466adde to 0x466d69d, (0x28bf bytes) 2024-09-21T15:33:05.186 INFO:teuthology.orchestra.run.smithi107.stdout:806 writesame 0xd402ccd thru 0xd40bafa data_size 0x1(0x8e2e bytes) 2024-09-21T15:33:05.188 INFO:teuthology.orchestra.run.smithi107.stdout:810 read 0x77075f1 thru 0x7716160 (0xeb70 bytes) 2024-09-21T15:33:05.189 INFO:teuthology.orchestra.run.smithi107.stdout:811 write 0x69703b4 thru 0x6977433 (0x7080 bytes) 2024-09-21T15:33:05.194 INFO:teuthology.orchestra.run.smithi107.stdout:812 punch from 0xa9bfff6 to 0xa9c5d9e, (0x5da8 bytes) 2024-09-21T15:33:05.195 INFO:teuthology.orchestra.run.smithi107.stdout:814 punch from 0xa0c3ffa to 0xa0d0fa0, (0xcfa6 bytes) 2024-09-21T15:33:05.195 INFO:teuthology.orchestra.run.smithi107.stdout:815 read 0xcdec447 thru 0xcdfb240 (0xedfa bytes) 2024-09-21T15:33:05.195 INFO:teuthology.orchestra.run.smithi107.stdout:816 punch from 0xc9f865 to 0xcab5fa, (0xbd95 bytes) 2024-09-21T15:33:05.195 INFO:teuthology.orchestra.run.smithi107.stdout:817 writesame 0x2ebf434 thru 0x2eccfd5 data_size 0x3(0xdba2 bytes) 2024-09-21T15:33:05.197 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:05.195+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:05.219 INFO:teuthology.orchestra.run.smithi107.stdout:818 write 0x827af58 thru 0x828a81b (0xf8c4 bytes) 2024-09-21T15:33:05.231 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.232 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.232 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.232 INFO:teuthology.orchestra.run.smithi107.stdout:823 read 0x974ddb2 thru 0x9751713 (0x3962 bytes) 2024-09-21T15:33:05.232 INFO:teuthology.orchestra.run.smithi107.stdout:824 write 0x7efd329 thru 0x7f096bf (0xc397 bytes) 2024-09-21T15:33:05.249 INFO:teuthology.orchestra.run.smithi107.stdout:825 read 0xb850c2a thru 0xb856640 (0x5a17 bytes) 2024-09-21T15:33:05.249 INFO:teuthology.orchestra.run.smithi107.stdout:826 write 0xc0f9c0e thru 0xc0fe6a7 (0x4a9a bytes) 2024-09-21T15:33:05.255 INFO:teuthology.orchestra.run.smithi107.stdout:827 read 0x707667d thru 0x707d15c (0x6ae0 bytes) 2024-09-21T15:33:05.256 INFO:teuthology.orchestra.run.smithi107.stdout:828 writesame 0x7cd95d2 thru 0x7ce1654 data_size 0x1(0x8083 bytes) 2024-09-21T15:33:05.264 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:05.265 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:05.280 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:05.292 INFO:teuthology.orchestra.run.smithi107.stdout:829 read 0x19894c3 thru 0x1998cf4 (0xf832 bytes) 2024-09-21T15:33:05.293 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.298 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:05.295+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:05.307 INFO:teuthology.orchestra.run.smithi107.stdout:831 trunc from 0xd40bafb to 0x8244822 2024-09-21T15:33:05.357 INFO:teuthology.orchestra.run.smithi107.stdout:832 trunc from 0x8244822 to 0x149cd 2024-09-21T15:33:05.399 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:05.395+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:05.436 INFO:teuthology.orchestra.run.smithi107.stdout:833 write 0xc134d62 thru 0xc1394b7 (0x4756 bytes) 2024-09-21T15:33:05.444 INFO:teuthology.orchestra.run.smithi107.stdout:834 read 0x3e52820 thru 0x3e584a6 (0x5c87 bytes) 2024-09-21T15:33:05.444 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.444 INFO:teuthology.orchestra.run.smithi107.stdout:837 write 0x919b3e3 thru 0x919b97c (0x59a bytes) 2024-09-21T15:33:05.450 INFO:teuthology.orchestra.run.smithi107.stdout:838 punch from 0x3362ed8 to 0x33634bb, (0x5e3 bytes) 2024-09-21T15:33:05.450 INFO:teuthology.orchestra.run.smithi107.stdout:839 write 0x89acbc5 thru 0x89b949a (0xc8d6 bytes) 2024-09-21T15:33:05.458 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:05.459 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:05.459 INFO:teuthology.orchestra.run.smithi107.stdout:840 write 0x1ef64a9 thru 0x1f031df (0xcd37 bytes) 2024-09-21T15:33:05.469 INFO:teuthology.orchestra.run.smithi107.stdout:841 write 0x734a378 thru 0x734d7cb (0x3454 bytes) 2024-09-21T15:33:05.475 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:842 read 0x39e030 thru 0x3a42ea (0x62bb bytes) 2024-09-21T15:33:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:846 read 0x2d58a20 thru 0x2d5eb4b (0x612c bytes) 2024-09-21T15:33:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:847 read 0x6a422bc thru 0x6a42e2e (0xb73 bytes) 2024-09-21T15:33:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:848 read 0x4520284 thru 0x452c23c (0xbfb9 bytes) 2024-09-21T15:33:05.501 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:05.499+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:05.501 INFO:teuthology.orchestra.run.smithi107.stdout:850 trunc from 0xc1394b8 to 0x36c3972 2024-09-21T15:33:05.520 INFO:teuthology.orchestra.run.smithi107.stdout:851 read 0x1352c2f thru 0x1362458 (0xf82a bytes) 2024-09-21T15:33:05.521 INFO:teuthology.orchestra.run.smithi107.stdout:853 read 0x134ee32 thru 0x134fe0b (0xfda bytes) 2024-09-21T15:33:05.527 INFO:teuthology.orchestra.run.smithi107.stdout:854 trunc from 0x36c3972 to 0x1835df0 2024-09-21T15:33:05.542 INFO:teuthology.orchestra.run.smithi107.stdout:855 read 0x10512a2 thru 0x105cdd1 (0xbb30 bytes) 2024-09-21T15:33:05.556 INFO:teuthology.orchestra.run.smithi107.stdout:856 trunc from 0x1835df0 to 0x6a0b6a8 2024-09-21T15:33:05.560 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:05.569 INFO:teuthology.orchestra.run.smithi107.stdout:857 writesame 0x422d1d4 thru 0x4238395 data_size 0x3d(0xb1c2 bytes) 2024-09-21T15:33:05.579 INFO:teuthology.orchestra.run.smithi107.stdout:859 punch from 0x1af5e49 to 0x1b03e34, (0xdfeb bytes) 2024-09-21T15:33:05.579 INFO:teuthology.orchestra.run.smithi107.stdout:860 punch from 0x413cbd4 to 0x4142e5e, (0x628a bytes) 2024-09-21T15:33:05.583 INFO:teuthology.orchestra.run.smithi107.stdout:861 trunc from 0x6a0b6a8 to 0x8472fe8 2024-09-21T15:33:05.593 INFO:teuthology.orchestra.run.smithi107.stdout:862 read 0x11ec9f9 thru 0x11f388c (0x6e94 bytes) 2024-09-21T15:33:05.594 INFO:teuthology.orchestra.run.smithi107.stdout:863 read 0x3ff5445 thru 0x4000699 (0xb255 bytes) 2024-09-21T15:33:05.601 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:05.601 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:05.601 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:05.599+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:05.615 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:05.620 INFO:teuthology.orchestra.run.smithi107.stdout:864 writesame 0xe953199 thru 0xe95fbc9 data_size 0x1(0xca31 bytes) 2024-09-21T15:33:05.620 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xeddf430 2024-09-21T15:33:05.620 INFO:teuthology.orchestra.run.smithi107.stdout:866 trunc from 0xe95fbca to 0xeddf430 2024-09-21T15:33:05.628 INFO:teuthology.orchestra.run.smithi107.stdout:868 punch from 0xb4ab3e3 to 0xb4b11a5, (0x5dc2 bytes) 2024-09-21T15:33:05.628 INFO:teuthology.orchestra.run.smithi107.stdout:869 write 0x4c25a8f thru 0x4c30103 (0xa675 bytes) 2024-09-21T15:33:05.632 INFO:teuthology.orchestra.run.smithi107.stdout:872 write 0x937241d thru 0x93732c7 (0xeab bytes) 2024-09-21T15:33:05.636 INFO:teuthology.orchestra.run.smithi107.stdout:873 write 0xdbc973 thru 0xdc55e4 (0x8c72 bytes) 2024-09-21T15:33:05.653 INFO:teuthology.orchestra.run.smithi107.stdout:874 trunc from 0xeddf430 to 0x50d9a10 2024-09-21T15:33:05.692 INFO:teuthology.orchestra.run.smithi107.stdout:876 write 0xe2125d7 thru 0xe21ec35 (0xc65f bytes) 2024-09-21T15:33:05.702 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:05.699+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:05.709 INFO:teuthology.orchestra.run.smithi107.stdout:877 trunc from 0xe21ec36 to 0x6914a2c 2024-09-21T15:33:05.721 INFO:teuthology.orchestra.run.smithi107.stdout:879 read 0x682cf7b thru 0x6839a7c (0xcb02 bytes) 2024-09-21T15:33:05.721 INFO:teuthology.orchestra.run.smithi107.stdout:880 read 0x2eeffbf thru 0x2ef7626 (0x7668 bytes) 2024-09-21T15:33:05.721 INFO:teuthology.orchestra.run.smithi107.stdout:882 write 0x5ae9cd thru 0x5bdc29 (0xf25d bytes) 2024-09-21T15:33:05.722 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:05.722 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:05.725 INFO:teuthology.orchestra.run.smithi107.stdout:883 write 0x4d12328 thru 0x4d18cbb (0x6994 bytes) 2024-09-21T15:33:05.727 INFO:teuthology.orchestra.run.smithi107.stdout:884 read 0x1ca0614 thru 0x1caf6f6 (0xf0e3 bytes) 2024-09-21T15:33:05.739 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:05.739 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:05.740 INFO:teuthology.orchestra.run.smithi107.stdout:885 write 0x932117f thru 0x93242f3 (0x3175 bytes) 2024-09-21T15:33:05.746 INFO:teuthology.orchestra.run.smithi107.stdout:886 trunc from 0x93242f4 to 0xb10f08e 2024-09-21T15:33:05.748 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:05.748 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:05.756 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:05.756 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:05.757 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.757 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.757 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.757 INFO:teuthology.orchestra.run.smithi107.stdout:890 punch from 0x650cecb to 0x651615b, (0x9290 bytes) 2024-09-21T15:33:05.757 INFO:teuthology.orchestra.run.smithi107.stdout:891 write 0x49e3785 thru 0x49e689b (0x3117 bytes) 2024-09-21T15:33:05.759 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.759 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.759 INFO:teuthology.orchestra.run.smithi107.stdout:894 writesame 0x970107e thru 0x9710e47 data_size 0x92(0xfdca bytes) 2024-09-21T15:33:05.765 INFO:teuthology.orchestra.run.smithi107.stdout:896 write 0x84fed5b thru 0x8508d1c (0x9fc2 bytes) 2024-09-21T15:33:05.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:05.769 INFO:teuthology.orchestra.run.smithi107.stdout:897 writesame 0x3440b89 thru 0x3441aac data_size 0x6(0xf24 bytes) 2024-09-21T15:33:05.774 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.774 INFO:teuthology.orchestra.run.smithi107.stdout:901 punch from 0x1a716b9 to 0x1a7e23c, (0xcb83 bytes) 2024-09-21T15:33:05.774 INFO:teuthology.orchestra.run.smithi107.stdout:902 read 0x9facf9e thru 0x9fb3e6d (0x6ed0 bytes) 2024-09-21T15:33:05.775 INFO:teuthology.orchestra.run.smithi107.stdout:904 writesame 0x7649ff thru 0x7657ff data_size 0xf(0xe01 bytes) 2024-09-21T15:33:05.775 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:05.775 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:05.776 INFO:teuthology.orchestra.run.smithi107.stdout:905 write 0x48990d2 thru 0x48a44bd (0xb3ec bytes) 2024-09-21T15:33:05.778 INFO:teuthology.orchestra.run.smithi107.stdout:907 write 0x64bac0a thru 0x64c4ce3 (0xa0da bytes) 2024-09-21T15:33:05.783 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.787 INFO:teuthology.orchestra.run.smithi107.stdout:910 trunc from 0xb10f08e to 0xdcfa8b1 2024-09-21T15:33:05.794 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:05.794 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:05.796 INFO:teuthology.orchestra.run.smithi107.stdout:911 trunc from 0xdcfa8b1 to 0xd082db8 2024-09-21T15:33:05.807 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:05.807+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:05.814 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:05.814 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:05.819 INFO:teuthology.orchestra.run.smithi107.stdout:912 trunc from 0xd082db8 to 0x2ffc231 2024-09-21T15:33:05.838 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:05.839 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:05.851 INFO:teuthology.orchestra.run.smithi107.stdout:914 writesame 0x7f59c74 thru 0x7f66fbb data_size 0x4(0xd348 bytes) 2024-09-21T15:33:05.851 INFO:teuthology.orchestra.run.smithi107.stdout:915 read 0x26a8833 thru 0x26adb7d (0x534b bytes) 2024-09-21T15:33:05.851 INFO:teuthology.orchestra.run.smithi107.stdout:916 read 0x4188bdc thru 0x418f937 (0x6d5c bytes) 2024-09-21T15:33:05.852 INFO:teuthology.orchestra.run.smithi107.stdout:917 read 0xd74faa thru 0xd823b3 (0xd40a bytes) 2024-09-21T15:33:05.852 INFO:teuthology.orchestra.run.smithi107.stdout:919 read 0xf44eb5 thru 0xf45eae (0xffa bytes) 2024-09-21T15:33:05.857 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:05.857 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:05.864 INFO:teuthology.orchestra.run.smithi107.stdout:920 write 0xa7af040 thru 0xa7b4275 (0x5236 bytes) 2024-09-21T15:33:05.870 INFO:teuthology.orchestra.run.smithi107.stdout:921 punch from 0x231c8bd to 0x2326181, (0x98c4 bytes) 2024-09-21T15:33:05.871 INFO:teuthology.orchestra.run.smithi107.stdout:922 punch from 0x175dc28 to 0x1765786, (0x7b5e bytes) 2024-09-21T15:33:05.871 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.871 INFO:teuthology.orchestra.run.smithi107.stdout:925 read 0x74bbf13 thru 0x74c3b15 (0x7c03 bytes) 2024-09-21T15:33:05.871 INFO:teuthology.orchestra.run.smithi107.stdout:928 writesame 0x5442a5c thru 0x544bb37 data_size 0x4(0x90dc bytes) 2024-09-21T15:33:05.879 INFO:teuthology.orchestra.run.smithi107.stdout:929 read 0x6b83ba5 thru 0x6b9239d (0xe7f9 bytes) 2024-09-21T15:33:05.880 INFO:teuthology.orchestra.run.smithi107.stdout:930 punch from 0x2f8dac to 0x3089fa, (0xfc4e bytes) 2024-09-21T15:33:05.880 INFO:teuthology.orchestra.run.smithi107.stdout:931 trunc from 0xa7b4276 to 0xb08840e 2024-09-21T15:33:05.895 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:05.895 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:05.896 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:05.903 INFO:teuthology.orchestra.run.smithi107.stdout:932 write 0xd8ea3f9 thru 0xd8f1824 (0x742c bytes) 2024-09-21T15:33:05.909 INFO:teuthology.orchestra.run.smithi107.stdout:933 write 0x6f3fa7a thru 0x6f45a98 (0x601f bytes) 2024-09-21T15:33:05.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:05.917 INFO:teuthology.orchestra.run.smithi107.stdout:935 punch from 0xe22a0 to 0xf0200, (0xdf60 bytes) 2024-09-21T15:33:05.917 INFO:teuthology.orchestra.run.smithi107.stdout:936 read 0x1a409e9 thru 0x1a42cd0 (0x22e8 bytes) 2024-09-21T15:33:05.918 INFO:teuthology.orchestra.run.smithi107.stdout:937 write 0xa04e87c thru 0xa053a1e (0x51a3 bytes) 2024-09-21T15:33:05.922 INFO:teuthology.orchestra.run.smithi107.stdout:938 trunc from 0xd8f1825 to 0xe1163b3 2024-09-21T15:33:05.929 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:05.903+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:05.932 INFO:teuthology.orchestra.run.smithi107.stdout:939 read 0x3f68bd6 thru 0x3f6fbc2 (0x6fed bytes) 2024-09-21T15:33:05.932 INFO:teuthology.orchestra.run.smithi107.stdout:940 writesame 0xe0e646d thru 0xe0e6aab data_size 0xd(0x63f bytes) 2024-09-21T15:33:05.944 INFO:teuthology.orchestra.run.smithi107.stdout:942 trunc from 0xe1163b3 to 0x8bc729c 2024-09-21T15:33:05.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:33:05.964 INFO:teuthology.orchestra.run.smithi107.stdout:943 punch from 0x292d5f5 to 0x292ed26, (0x1731 bytes) 2024-09-21T15:33:05.964 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.964 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:05.964 INFO:teuthology.orchestra.run.smithi107.stdout:947 read 0x65de44f thru 0x65e78ae (0x9460 bytes) 2024-09-21T15:33:05.964 INFO:teuthology.orchestra.run.smithi107.stdout:948 write 0x13f3e78 thru 0x13f469c (0x825 bytes) 2024-09-21T15:33:05.968 INFO:teuthology.orchestra.run.smithi107.stdout:950 trunc from 0x8bc729c to 0x9300258 2024-09-21T15:33:05.997 INFO:teuthology.orchestra.run.smithi107.stdout:951 writesame 0xe3ad516 thru 0xe3b9eaf data_size 0x41(0xc99a bytes) 2024-09-21T15:33:06.005 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:06.003+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:06.010 INFO:teuthology.orchestra.run.smithi107.stdout:952 write 0xea88c89 thru 0xea8bfdb (0x3353 bytes) 2024-09-21T15:33:06.013 INFO:teuthology.orchestra.run.smithi107.stdout:954 read 0x5a2d67e thru 0x5a37b2c (0xa4af bytes) 2024-09-21T15:33:06.015 INFO:teuthology.orchestra.run.smithi107.stdout:955 read 0x31f78d4 thru 0x31f90ff (0x182c bytes) 2024-09-21T15:33:06.015 INFO:teuthology.orchestra.run.smithi107.stdout:956 read 0x2c68ac1 thru 0x2c748ae (0xbdee bytes) 2024-09-21T15:33:06.015 INFO:teuthology.orchestra.run.smithi107.stdout:957 write 0x7aaf4ef thru 0x7abee96 (0xf9a8 bytes) 2024-09-21T15:33:06.020 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:06.020 INFO:teuthology.orchestra.run.smithi107.stdout:961 read 0xc522839 thru 0xc530e5f (0xe627 bytes) 2024-09-21T15:33:06.036 INFO:teuthology.orchestra.run.smithi107.stdout:963 trunc from 0xea8bfdc to 0x3b88728 2024-09-21T15:33:06.047 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:06.047 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:06.048 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:06.048 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:06.048 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:06.057 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:06.057 INFO:teuthology.orchestra.run.smithi107.stdout:965 read 0x879a02 thru 0x887826 (0xde25 bytes) 2024-09-21T15:33:06.057 INFO:teuthology.orchestra.run.smithi107.stdout:967 read 0x35d42b thru 0x36689a (0x9470 bytes) 2024-09-21T15:33:06.059 INFO:teuthology.orchestra.run.smithi107.stdout:968 read 0x2b8bea9 thru 0x2b942f2 (0x844a bytes) 2024-09-21T15:33:06.059 INFO:teuthology.orchestra.run.smithi107.stdout:969 punch from 0xebbf1a to 0xec891c, (0xca02 bytes) 2024-09-21T15:33:06.064 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:06.080 INFO:teuthology.orchestra.run.smithi107.stdout:970 writesame 0xa958d39 thru 0xa9591f8 data_size 0x4(0x4c0 bytes) 2024-09-21T15:33:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:971 write 0xd744bbe thru 0xd747022 (0x2465 bytes) 2024-09-21T15:33:06.098 INFO:teuthology.orchestra.run.smithi107.stdout:972 write 0xc284474 thru 0xc28e2c9 (0x9e56 bytes) 2024-09-21T15:33:06.106 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:06.103+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:06.106 INFO:teuthology.orchestra.run.smithi107.stdout:973 trunc from 0xd747023 to 0xc24b0c3 2024-09-21T15:33:06.120 INFO:teuthology.orchestra.run.smithi107.stdout:974 clone 11 order 25 su 8388608 sc 6 2024-09-21T15:33:06.201 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:06.201 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:06.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.663261124000002, 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:33:06.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.662689047999997, 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:33:06.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:897 3.1f 3:ff84ad69:::rbd_data.11911fd826fd.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.194300015, 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061454999999999999, 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 897 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.211 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004191+0000", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004190+0000", 2024-09-21T15:33:06.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004198+0000", 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7639999999999994e-06 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004247+0000", 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8167999999999997e-05 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004260+0000", 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3106e-05 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004323+0000", 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3482e-05 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004424+0000", 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101192 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.004750+0000", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032567099999999998 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.006929+0000", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021786119999999999 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010188+0000", 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032598100000000001 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010282+0000", 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3350000000000006e-05 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.010337+0000", 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4744999999999998e-05 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:946 3.1b 3:db55d07f:::rbd_data.11911fd826fd.000000000000001e:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.216 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.183394839, 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062820289999999997, 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 946 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:06.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015096+0000", 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015095+0000", 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015102+0000", 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7009999999999999e-06 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015116+0000", 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.412e-05 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0429999999999999e-05 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015450+0000", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031393000000000003 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015616+0000", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016547900000000001 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.021378+0000", 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057623930000000002 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.220 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:981 3.1 3:812dd4d6:::rbd_data.11911fd826fd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.16620575, 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073781309999999996, 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 981 2024-09-21T15:33:06.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032284+0000", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032291+0000", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032306+0000", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5162e-05 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032317+0000", 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0641e-05 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032391+0000", 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3563e-05 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032531+0000", 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014023900000000001 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032886+0000", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035541499999999998 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035001+0000", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021150409999999998 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039546+0000", 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045443339999999997 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039615+0000", 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8936000000000005e-05 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039663+0000", 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8866000000000003e-05 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.226 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:984 3.6 3:630b821a:::rbd_data.11911fd826fd.0000000000000033:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.165731678, 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006946069, 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:06.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032758+0000", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032765+0000", 2024-09-21T15:33:06.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4590000000000001e-06 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032780+0000", 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032789+0000", 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4339999999999998e-06 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032836+0000", 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6891000000000002e-05 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032919+0000", 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3238000000000005e-05 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033108+0000", 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000188803 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035121+0000", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020131900000000002 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044746530000000003 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039653+0000", 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6486000000000001e-05 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039705+0000", 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2935000000000001e-05 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:985 3.17 3:eca73502:::rbd_data.11911fd826fd.0000000000000034:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.165322305, 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0094186879999999997, 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033168+0000", 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033175+0000", 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1729999999999998e-06 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033194+0000", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9383999999999999e-05 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033205+0000", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0947e-05 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033266+0000", 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0551999999999999e-05 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033373+0000", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107155 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033600+0000", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022703799999999999 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035620+0000", 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020200050000000001 2024-09-21T15:33:06.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042474+0000", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068545359999999996 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042537+0000", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2778000000000001e-05 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042587+0000", 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0272999999999998e-05 2024-09-21T15:33:06.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:982 3.10 e27/21 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.165209116, 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063444290000000004, 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033281+0000", 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033295+0000", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3361e-05 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033306+0000", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1268e-05 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033313+0000", 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6450000000000005e-06 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033365+0000", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1208999999999997e-05 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033391+0000", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6353e-05 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039585+0000", 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061938990000000001 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039626+0000", 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:987 3.1f 3:fa7a7055:::rbd_data.11911fd826fd.0000000000000036:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.163502188000001, 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077071170000000003, 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034988+0000", 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034996+0000", 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2360000000000004e-06 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035015+0000", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9072e-05 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035028+0000", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2668e-05 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035078+0000", 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0667999999999997e-05 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035162+0000", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3707000000000003e-05 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035389+0000", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022708 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.037465+0000", 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020759379999999998 2024-09-21T15:33:06.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042600+0000", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051350750000000002 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042653+0000", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3196999999999999e-05 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042696+0000", 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2800999999999999e-05 2024-09-21T15:33:06.247 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:988 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.163295848000001, 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073861910000000003, 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035194+0000", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035203+0000", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2059999999999996e-06 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035222+0000", 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9320000000000001e-05 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035233+0000", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0793e-05 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035310+0000", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6808000000000001e-05 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035479+0000", 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016872999999999999 2024-09-21T15:33:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035835+0000", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035607999999999999 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039777+0000", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039426979999999997 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042513+0000", 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002735786 2024-09-21T15:33:06.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042550+0000", 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7211e-05 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042581+0000", 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1031000000000002e-05 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3169 3.b e30/21 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head v 30'28, mlcod=29'25)", 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.3137160039999998, 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010485235000000001, 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884773+0000", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884789+0000", 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.556e-05 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884822+0000", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3189999999999999e-05 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884860+0000", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8166e-05 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884989+0000", 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012823500000000001 2024-09-21T15:33:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885053+0000", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4503000000000002e-05 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895205+0000", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010151415 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895260+0000", 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5736000000000003e-05 2024-09-21T15:33:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3170 3.17 3:eb82c48b:::rbd_data.1191ab8bc3c6.0000000000000001:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.3114806479999999, 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095184950000000001, 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3170 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887007+0000", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887024+0000", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.647e-05 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887072+0000", 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8038999999999999e-05 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887096+0000", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3777e-05 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887229+0000", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133805 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887459+0000", 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229535 2024-09-21T15:33:06.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887904+0000", 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044505 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892712+0000", 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048083960000000004 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896348+0000", 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036350990000000001 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896471+0000", 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012388300000000001 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896529+0000", 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7491000000000002e-05 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.263 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3172 3.b e30/21 3:d0ec678c:::rbd_data.1191ab8bc3c6.0000000000000003:head v 30'30, mlcod=29'25)", 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.3064292660000003, 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073246989999999996, 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:06.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892061+0000", 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892072+0000", 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1301000000000001e-05 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892087+0000", 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.504e-05 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892097+0000", 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969999999999993e-06 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892204+0000", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106783 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892258+0000", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4014000000000002e-05 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899269+0000", 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070109120000000002 2024-09-21T15:33:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899386+0000", 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117793 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3173 3.1 3:84db22e7:::rbd_data.1191ab8bc3c6.0000000000000004:head [stat,truncate 6032748] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.3062187820000002, 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071419710000000004, 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3173 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892269+0000", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892286+0000", 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6826000000000001e-05 2024-09-21T15:33:06.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892323+0000", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7292999999999997e-05 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892345+0000", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2186000000000001e-05 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892469+0000", 2024-09-21T15:33:06.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000123566 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892666+0000", 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019718900000000001 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893107+0000", 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044130900000000003 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895184+0000", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020761920000000001 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899255+0000", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040709980000000002 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899355+0000", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100543 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899414+0000", 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9119000000000001e-05 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4150 3.1f 3:fe00fe50:::rbd_data.1191cae494be.0000000000000002:a [sparse-read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.5043381389999997, 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060212410000000001, 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4150 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694153+0000", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694155+0000", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9269999999999999e-06 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694160+0000", 2024-09-21T15:33:06.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5720000000000002e-06 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.694165+0000", 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5600000000000004e-06 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.699958+0000", 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00579293 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.700050+0000", 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2114999999999995e-05 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.700174+0000", 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124459 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:06.278 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.4888235659999998, 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:06.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.4785012709999998, 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:06.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.288 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5510040460000001, 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:06.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:06.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:06.294 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5170926119999999, 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:06.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:06.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5749 3.6 3:65672256:::rbd_header.11917ba4f9b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T15:33:06.298 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.2844907769999998, 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063958840000000001, 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5749 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:06.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.913998+0000", 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914014+0000", 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6646e-05 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914047+0000", 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2270000000000001e-05 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914070+0000", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3572999999999999e-05 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914205+0000", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013493899999999999 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914476+0000", 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027055800000000001 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916280+0000", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018040560000000001 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.919876+0000", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035958740000000002 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920299+0000", 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042360999999999999 2024-09-21T15:33:06.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920352+0000", 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.253e-05 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920396+0000", 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4418000000000001e-05 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.305 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:06.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:06.307 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:06.303+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:06.408 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:06.407+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:06.472 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:06.472 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:06.473 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:06.473 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:06.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:06.510 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:06.511+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:06.611 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:06.611+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:06.652 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:06.653 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:06.653 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:06.653 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:06.653 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:06.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:06.712 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:06.711+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:06.794 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:06.794 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.059930239, 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.799 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:641 3.0 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write 490537~26655] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.318013125, 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003364204, 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 641 2024-09-21T15:33:06.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473719+0000", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473716+0000", 2024-09-21T15:33:06.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473753+0000", 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6430000000000002e-05 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473790+0000", 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7536000000000001e-05 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473827+0000", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6779999999999997e-05 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.473923+0000", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6143000000000004e-05 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474178+0000", 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000255374 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474426+0000", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024796799999999999 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.475922+0000", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001495515 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476887+0000", 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00096475599999999999 2024-09-21T15:33:06.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476972+0000", 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5562999999999994e-05 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.477084+0000", 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000111319 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.805 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.249518617, 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:33:06.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.811 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:688 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.239886034, 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032222280000000002, 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 688 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.812 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551847+0000", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551845+0000", 2024-09-21T15:33:06.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551852+0000", 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6499999999999999e-06 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551870+0000", 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7853e-05 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.551882+0000", 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2459999999999999e-05 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554681+0000", 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002798189 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.554810+0000", 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012915499999999999 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.555069+0000", 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025897799999999999 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:716 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213962815, 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034336240000000001, 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 716 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577770+0000", 2024-09-21T15:33:06.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577768+0000", 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577777+0000", 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4470000000000006e-06 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577796+0000", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9769999999999999e-05 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577811+0000", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4686e-05 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580639+0000", 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028279590000000001 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.580881+0000", 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000241955 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581203+0000", 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000322267 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.820 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213907468, 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.824 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213827032999999, 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:33:06.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:33:06.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213781889, 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:33:06.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:33:06.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:33:06.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213603661, 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:33:06.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:33:06.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.213530011, 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.838 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:33:06.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:06.842 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.795523184, 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:06.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.846 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:943 3.1a e27/21 3:5cf65bc3:::rbd_data.11911fd826fd.0000000000000015:head v 27'13, mlcod=27'11)", 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.776629885, 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034511770000000002, 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:06.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015103+0000", 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015101+0000", 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015110+0000", 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6519999999999995e-06 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015127+0000", 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7030000000000001e-05 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015136+0000", 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8440000000000004e-06 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015199+0000", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2786999999999994e-05 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.015229+0000", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9927e-05 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018501+0000", 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032721299999999998 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.018554+0000", 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2970999999999998e-05 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.8 epoch 27/21 rep_tid 41 entries 27'3 (0'0) error 3:15312f39:::rbd_data.11911fd826fd.0000000000000020:head by client.4497.0:950 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.774533635000001, 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037814699999999999, 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017205+0000", 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7449999999999998e-06 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017223+0000", 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7626999999999999e-05 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017241+0000", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.749e-05 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017302+0000", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1802000000000002e-05 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.020980+0000", 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036779909999999998 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:988 3.6 e27/21 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head v 27'10, mlcod=26'6)", 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.755776526, 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036395920000000001, 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:06.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035955+0000", 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035964+0000", 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035980+0000", 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5855e-05 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035988+0000", 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8650000000000001e-06 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036064+0000", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6191000000000003e-05 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036088+0000", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3470000000000001e-05 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039519+0000", 2024-09-21T15:33:06.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034318809999999999 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6165e-05 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5390 3.6 e33/21 3:65672256:::rbd_header.11917ba4f9b:head v 33'40, mlcod=33'38)", 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.1640141990000004, 2024-09-21T15:33:06.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032420679999999999, 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627718+0000", 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627716+0000", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627732+0000", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.571e-05 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627765+0000", 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3210000000000002e-05 2024-09-21T15:33:06.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627779+0000", 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3993e-05 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627850+0000", 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1402000000000005e-05 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.627884+0000", 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3318000000000002e-05 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.630912+0000", 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030284999999999999 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.630960+0000", 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8267999999999999e-05 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.863 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5584 3.1a e33/21 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head v 33'40, mlcod=32'38)", 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.739523106, 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053598999999999999, 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052208+0000", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056205+0000", 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003997202 2024-09-21T15:33:06.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056221+0000", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6167e-05 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056231+0000", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.55e-06 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056297+0000", 2024-09-21T15:33:06.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6231999999999999e-05 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056329+0000", 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1977999999999998e-05 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057476+0000", 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011471280000000001 2024-09-21T15:33:06.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057569+0000", 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3178999999999997e-05 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.868 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5749 3.6 e34/21 3:65672256:::rbd_header.11917ba4f9b:head v 34'68, mlcod=33'66)", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.916681+0000", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.8750515449999998, 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034185029999999998, 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916681+0000", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916681+0000", 2024-09-21T15:33:06.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916681+0000", 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916687+0000", 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2779999999999999e-06 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916704+0000", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7195999999999999e-05 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916716+0000", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2388999999999999e-05 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916879+0000", 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016282300000000001 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916936+0000", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6645999999999999e-05 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920042+0000", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031057720000000001 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920099+0000", 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7765e-05 2024-09-21T15:33:06.872 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7045 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.865726+0000", 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.92600646799999997, 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003271173, 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7045 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865726+0000", 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865726+0000", 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865726+0000", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865730+0000", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4469999999999999e-06 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865736+0000", 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.392e-06 2024-09-21T15:33:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.865741+0000", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9690000000000002e-06 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866508+0000", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00076699199999999998 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866613+0000", 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000104219 2024-09-21T15:33:06.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866913+0000", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029991399999999997 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867941+0000", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010287950000000001 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868943+0000", 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001001828 2024-09-21T15:33:06.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868973+0000", 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9830999999999999e-05 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868997+0000", 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4253e-05 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7071 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88697513699999997, 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041211169999999997, 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7071 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:06.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904762+0000", 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904770+0000", 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5879999999999998e-06 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904777+0000", 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6170000000000003e-06 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905465+0000", 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00068793299999999995 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905513+0000", 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7842e-05 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905832+0000", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031906099999999999 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908509+0000", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026772739999999999 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908784+0000", 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027538999999999998 2024-09-21T15:33:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908833+0000", 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8958000000000001e-05 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908879+0000", 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5105999999999998e-05 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.884 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7075 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88090688500000003, 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045190079999999997, 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7075 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:06.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910825+0000", 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910830+0000", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910837+0000", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2740000000000002e-06 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910842+0000", 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3789999999999997e-06 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911544+0000", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00070163500000000004 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911629+0000", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4837000000000001e-05 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912033+0000", 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040393900000000002 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915180+0000", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031479189999999999 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915295+0000", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114585 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915324+0000", 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8923000000000001e-05 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:06.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915345+0000", 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0659e-05 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:06.890 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:06.891 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:06.892 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:06.887+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:06.956 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:33:07.036 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:07.035+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:07.071 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:07.071 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:07.071 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:07.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:07.087 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone10 from 0xc24b0c3 (overlap 0x149cd) to 0x62f0 2024-09-21T15:33:07.128 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:07.127+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:07.193 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:07.191+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:07.250 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T15:33:07.255 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.255 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:07.255 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:07.255 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:07.255 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:07.260 INFO:teuthology.orchestra.run.smithi107.stdout:976 write 0x8e81fe2 thru 0x8e85c21 (0x3c40 bytes) 2024-09-21T15:33:07.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:07.272 INFO:teuthology.orchestra.run.smithi107.stdout:977 read 0x8541fd8 thru 0x8546f5f (0x4f88 bytes) 2024-09-21T15:33:07.280 INFO:teuthology.orchestra.run.smithi107.stdout:978 trunc from 0xc24b0c3 to 0x680d4f1 2024-09-21T15:33:07.294 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:07.291+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:07.313 INFO:teuthology.orchestra.run.smithi107.stdout:979 trunc from 0x680d4f1 to 0xe6958a0 2024-09-21T15:33:07.321 INFO:teuthology.orchestra.run.smithi107.stdout:980 write 0xcb0da97 thru 0xcb1a1ca (0xc734 bytes) 2024-09-21T15:33:07.326 INFO:teuthology.orchestra.run.smithi107.stdout:981 punch from 0xe4df817 to 0xe4dff88, (0x771 bytes) 2024-09-21T15:33:07.326 INFO:teuthology.orchestra.run.smithi107.stdout:982 read 0x64614cb thru 0x646dd67 (0xc89d bytes) 2024-09-21T15:33:07.327 INFO:teuthology.orchestra.run.smithi107.stdout:983 read 0x933b84f thru 0x934b2df (0xfa91 bytes) 2024-09-21T15:33:07.327 INFO:teuthology.orchestra.run.smithi107.stdout:985 read 0xce2c976 thru 0xce3c167 (0xf7f2 bytes) 2024-09-21T15:33:07.327 INFO:teuthology.orchestra.run.smithi107.stdout:986 read 0xdd97c09 thru 0xdda37c3 (0xbbbb bytes) 2024-09-21T15:33:07.327 INFO:teuthology.orchestra.run.smithi107.stdout:988 read 0xbd10119 thru 0xbd16628 (0x6510 bytes) 2024-09-21T15:33:07.330 INFO:teuthology.orchestra.run.smithi107.stdout:991 trunc from 0xe6958a0 to 0xd38ab56 2024-09-21T15:33:07.343 INFO:teuthology.orchestra.run.smithi107.stdout:995 trunc from 0xd38ab56 to 0x9d75d5a 2024-09-21T15:33:07.362 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.362 INFO:teuthology.orchestra.run.smithi107.stdout:998 write 0x97f063b thru 0x97fc2b8 (0xbc7e bytes) 2024-09-21T15:33:07.379 INFO:teuthology.orchestra.run.smithi107.stdout:999 write 0xdaae658 thru 0xdab8db2 (0xa75b bytes) 2024-09-21T15:33:07.385 INFO:teuthology.orchestra.run.smithi107.stdout:1000 read 0x7345178 thru 0x7350713 (0xb59c bytes) 2024-09-21T15:33:07.395 INFO:teuthology.orchestra.run.smithi107.stdout:1003 write 0xedaf116 thru 0xedaf14d (0x38 bytes) 2024-09-21T15:33:07.396 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:07.395+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:07.407 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.407 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:07.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:641 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'8, mlcod=26'6)", 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.931355765999999, 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001951022, 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474657+0000", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474747+0000", 2024-09-21T15:33:07.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9559000000000001e-05 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474777+0000", 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.991e-05 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474790+0000", 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3246e-05 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474904+0000", 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011387600000000001 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474973+0000", 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8847e-05 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476526+0000", 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015536180000000001 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476610+0000", 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3931999999999995e-05 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.862073430000001, 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:33:07.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:33:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:738 3.9 e26/21 3:9661011e:::rbd_data.11911fd826fd.0000000000000032:head v 26'5, mlcod=26'3)", 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.818526113000001, 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017551730000000001, 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587489+0000", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587487+0000", 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587502+0000", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5460000000000001e-05 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587531+0000", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.836e-05 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587551+0000", 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0299999999999999e-05 2024-09-21T15:33:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587650+0000", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9412000000000003e-05 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.587709+0000", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8536000000000003e-05 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.589182+0000", 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001473495 2024-09-21T15:33:07.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.589244+0000", 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1729999999999993e-05 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3130 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'17, mlcod=28'14)", 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.582338311999999, 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018694930000000001, 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823676+0000", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823685+0000", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823711+0000", 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6106e-05 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823726+0000", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4644000000000001e-05 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823829+0000", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102505 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823867+0000", 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8606999999999998e-05 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825433+0000", 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015654810000000001 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825546+0000", 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113416 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.425 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391157159, 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:33:07.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:33:07.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:33:07.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.381367066999999, 2024-09-21T15:33:07.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.434 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.367413027, 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:07.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:33:07.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.359245882, 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:33:07.443 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.358499482999999, 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:07.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.340108507, 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:07.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:33:07.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.452 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.279242567000001, 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:07.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.456 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.267572231000001, 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:33:07.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:33:07.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7045 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'170, mlcod=35'167)", 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5390269060000001, 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001819846, 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.461 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866987+0000", 2024-09-21T15:33:07.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866995+0000", 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0930000000000003e-06 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867006+0000", 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0781999999999999e-05 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867035+0000", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9026e-05 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867077+0000", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1337000000000001e-05 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867099+0000", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2486000000000001e-05 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868767+0000", 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00166782 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868808+0000", 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1032000000000003e-05 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7071 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'176, mlcod=35'173)", 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:07.465 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.5001042999999998, 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027688610000000001, 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:07.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905910+0000", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905929+0000", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9409999999999999e-05 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905943+0000", 2024-09-21T15:33:07.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4109e-05 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905950+0000", 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905995+0000", 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4925000000000001e-05 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.906020+0000", 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4771000000000001e-05 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908648+0000", 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002627622 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908680+0000", 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1795999999999998e-05 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.469 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7075 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'178, mlcod=35'175)", 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.493887701, 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030847729999999999, 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912134+0000", 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5449999999999998e-06 2024-09-21T15:33:07.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912146+0000", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1834999999999999e-05 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912153+0000", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7819999999999996e-06 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912186+0000", 2024-09-21T15:33:07.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3667999999999997e-05 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912199+0000", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2835e-05 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915184+0000", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029846120000000002 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915212+0000", 2024-09-21T15:33:07.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8242000000000002e-05 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30909713300000002, 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30818627700000001, 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:07.482 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30722034999999998, 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:33:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:33:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7329 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'206, mlcod=36'204)", 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.29794692900000003, 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003080213, 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108067+0000", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108077+0000", 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7990000000000005e-06 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108098+0000", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1523999999999999e-05 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108109+0000", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0567999999999999e-05 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108153+0000", 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4187999999999998e-05 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108175+0000", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2089000000000001e-05 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111116+0000", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029405490000000002 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111148+0000", 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2619000000000001e-05 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.490 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.29701636799999998, 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:07.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:33:07.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:33:07.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.495 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:07.496 INFO:teuthology.orchestra.run.smithi107.stdout:1006 trunc from 0xedaf14e to 0x4d0abb8 2024-09-21T15:33:07.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:07.497 INFO:teuthology.orchestra.run.smithi107.stdout:1009 trunc from 0x4d0abb8 to 0x424968e 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1010 write 0x1e635c3 thru 0x1e67c8f (0x46cd bytes) 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1011 trunc from 0x424968e to 0x32ac4c1 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1012 read 0x2be28d0 thru 0x2be3e5e (0x158f bytes) 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1013 read 0x152f5ff thru 0x1537d14 (0x8716 bytes) 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1015 writesame 0x18fb54e thru 0x18fc8ee data_size 0x19(0x13a1 bytes) 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1016 punch from 0x279c402 to 0x27aa6eb, (0xe2e9 bytes) 2024-09-21T15:33:07.498 INFO:teuthology.orchestra.run.smithi107.stdout:1017 punch from 0x26abdc3 to 0x26b6ca5, (0xaee2 bytes) 2024-09-21T15:33:07.498 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:07.495+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:07.509 INFO:teuthology.orchestra.run.smithi107.stdout:1018 write 0xe8fa4ae thru 0xe8ff2a3 (0x4df6 bytes) 2024-09-21T15:33:07.515 INFO:teuthology.orchestra.run.smithi107.stdout:1021 read 0x5026235 thru 0x502c363 (0x612f bytes) 2024-09-21T15:33:07.516 INFO:teuthology.orchestra.run.smithi107.stdout:1022 writesame 0xe077428 thru 0xe0844e7 data_size 0xa0(0xd0c0 bytes) 2024-09-21T15:33:07.522 INFO:teuthology.orchestra.run.smithi107.stdout:1023 read 0xb527ec1 thru 0xb53696e (0xeaae bytes) 2024-09-21T15:33:07.523 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.523 INFO:teuthology.orchestra.run.smithi107.stdout:1026 write 0xa9a82d4 thru 0xa9b1392 (0x90bf bytes) 2024-09-21T15:33:07.529 INFO:teuthology.orchestra.run.smithi107.stdout:1027 write 0xc35bfcc thru 0xc369393 (0xd3c8 bytes) 2024-09-21T15:33:07.534 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:1033 read 0x6c064b thru 0x6cc298 (0xbc4e bytes) 2024-09-21T15:33:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:1034 punch from 0xd80d630 to 0xd816c93, (0x9663 bytes) 2024-09-21T15:33:07.535 INFO:teuthology.orchestra.run.smithi107.stdout:1039 read 0x23cf337 thru 0x23dbad3 (0xc79d bytes) 2024-09-21T15:33:07.536 INFO:teuthology.orchestra.run.smithi107.stdout:1040 write 0x2304663 thru 0x230e28f (0x9c2d bytes) 2024-09-21T15:33:07.540 INFO:teuthology.orchestra.run.smithi107.stdout:1041 write 0x32db426 thru 0x32eaa43 (0xf61e bytes) 2024-09-21T15:33:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:1042 read 0x5832cab thru 0x583f03d (0xc393 bytes) 2024-09-21T15:33:07.543 INFO:teuthology.orchestra.run.smithi107.stdout:1043 punch from 0x719d906 to 0x71a3a7d, (0x6177 bytes) 2024-09-21T15:33:07.543 INFO:teuthology.orchestra.run.smithi107.stdout:1044 read 0x65486cc thru 0x6548afb (0x430 bytes) 2024-09-21T15:33:07.543 INFO:teuthology.orchestra.run.smithi107.stdout:1045 write 0xcddc208 thru 0xcde4239 (0x8032 bytes) 2024-09-21T15:33:07.549 INFO:teuthology.orchestra.run.smithi107.stdout:1046 read 0x474096c thru 0x4741531 (0xbc6 bytes) 2024-09-21T15:33:07.550 INFO:teuthology.orchestra.run.smithi107.stdout:1047 writesame 0x6943c2a thru 0x6953b40 data_size 0x13(0xff17 bytes) 2024-09-21T15:33:07.556 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.556 INFO:teuthology.orchestra.run.smithi107.stdout:1049 punch from 0x5912931 to 0x5920dd0, (0xe49f bytes) 2024-09-21T15:33:07.556 INFO:teuthology.orchestra.run.smithi107.stdout:1050 write 0x3cc4fb3 thru 0x3ccbebf (0x6f0d bytes) 2024-09-21T15:33:07.560 INFO:teuthology.orchestra.run.smithi107.stdout:1051 punch from 0x3ecc1e9 to 0x3ed8c40, (0xca57 bytes) 2024-09-21T15:33:07.560 INFO:teuthology.orchestra.run.smithi107.stdout:1053 writesame 0xd88b5dd thru 0xd88d943 data_size 0x13(0x2367 bytes) 2024-09-21T15:33:07.567 INFO:teuthology.orchestra.run.smithi107.stdout:1054 write 0x350eef0 thru 0x35150f6 (0x6207 bytes) 2024-09-21T15:33:07.587 INFO:teuthology.orchestra.run.smithi107.stdout:1056 trunc from 0xe8ff2a4 to 0x4484e45 2024-09-21T15:33:07.597 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:07.595+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:07.612 INFO:teuthology.orchestra.run.smithi107.stdout:1057 read 0x25749fb thru 0x2574d67 (0x36d bytes) 2024-09-21T15:33:07.631 INFO:teuthology.orchestra.run.smithi107.stdout:1058 trunc from 0x4484e45 to 0xcfa405e 2024-09-21T15:33:07.639 INFO:teuthology.orchestra.run.smithi107.stdout:1059 write 0x33f6d08 thru 0x33fd34f (0x6648 bytes) 2024-09-21T15:33:07.642 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.643 INFO:teuthology.orchestra.run.smithi107.stdout:1061 writesame 0x4ad9c67 thru 0x4adbfd6 data_size 0x48(0x2370 bytes) 2024-09-21T15:33:07.646 INFO:teuthology.orchestra.run.smithi107.stdout:1062 writesame 0xcbf88a0 thru 0xcbf9070 data_size 0x3(0x7d1 bytes) 2024-09-21T15:33:07.647 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.647 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:07.647 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:07.647 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:07.653 INFO:teuthology.orchestra.run.smithi107.stdout:1063 read 0x76016cd thru 0x76104f0 (0xee24 bytes) 2024-09-21T15:33:07.654 INFO:teuthology.orchestra.run.smithi107.stdout:1064 write 0x753d7c9 thru 0x753ef01 (0x1739 bytes) 2024-09-21T15:33:07.665 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:07.670 INFO:teuthology.orchestra.run.smithi107.stdout:1065 writesame 0xea74174 thru 0xea7cac6 data_size 0x5(0x8953 bytes) 2024-09-21T15:33:07.670 INFO:teuthology.orchestra.run.smithi107.stdout:1067 read 0x9462611 thru 0x946d2c6 (0xacb6 bytes) 2024-09-21T15:33:07.670 INFO:teuthology.orchestra.run.smithi107.stdout:1069 read 0x93f627e thru 0x940314b (0xcece bytes) 2024-09-21T15:33:07.678 INFO:teuthology.orchestra.run.smithi107.stdout:1070 trunc from 0xea7cac7 to 0xb1f7645 2024-09-21T15:33:07.698 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:07.695+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:1071 writesame 0xddca920 thru 0xddd808e data_size 0x1(0xd76f bytes) 2024-09-21T15:33:07.710 INFO:teuthology.orchestra.run.smithi107.stdout:1072 writesame 0xc27a070 thru 0xc27e334 data_size 0x1(0x42c5 bytes) 2024-09-21T15:33:07.748 INFO:teuthology.orchestra.run.smithi107.stdout:1073 trunc from 0xddd808f to 0x650dabd 2024-09-21T15:33:07.790 INFO:teuthology.orchestra.run.smithi107.stdout:1074 writesame 0x902038d thru 0x9021d11 data_size 0x1(0x1985 bytes) 2024-09-21T15:33:07.792 INFO:teuthology.orchestra.run.smithi107.stdout:1075 punch from 0x51a4bb4 to 0x51af339, (0xa785 bytes) 2024-09-21T15:33:07.792 INFO:teuthology.orchestra.run.smithi107.stdout:1076 write 0x19ce0d thru 0x1a0fda (0x41ce bytes) 2024-09-21T15:33:07.799 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:07.795+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:07.800 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:07.801 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:07.818 INFO:teuthology.orchestra.run.smithi107.stdout:1077 write 0xdd704fd thru 0xdd80439 (0xff3d bytes) 2024-09-21T15:33:07.820 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:07.825 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:07.825 INFO:teuthology.orchestra.run.smithi107.stdout:1081 read 0x2b7581d thru 0x2b75820 (0x4 bytes) 2024-09-21T15:33:07.826 INFO:teuthology.orchestra.run.smithi107.stdout:1082 punch from 0x843e00d to 0x844bd19, (0xdd0c bytes) 2024-09-21T15:33:07.836 INFO:teuthology.orchestra.run.smithi107.stdout:1083 writesame 0xe257b5e thru 0xe260688 data_size 0x1(0x8b2b bytes) 2024-09-21T15:33:07.852 INFO:teuthology.orchestra.run.smithi107.stdout:1084 trunc from 0xe260689 to 0x34dcd04 2024-09-21T15:33:07.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:07.889 INFO:teuthology.orchestra.run.smithi107.stdout:1085 write 0x6e0bc57 thru 0x6e12a62 (0x6e0c bytes) 2024-09-21T15:33:07.892 INFO:teuthology.orchestra.run.smithi107.stdout:1086 read 0x4742b0a thru 0x4751c95 (0xf18c bytes) 2024-09-21T15:33:07.900 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:07.899+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:07.902 INFO:teuthology.orchestra.run.smithi107.stdout:1087 writesame 0x8a94c16 thru 0x8aa0a94 data_size 0x1(0xbe7f bytes) 2024-09-21T15:33:07.904 INFO:teuthology.orchestra.run.smithi107.stdout:1088 read 0x76e265 thru 0x775bd7 (0x7973 bytes) 2024-09-21T15:33:07.904 INFO:teuthology.orchestra.run.smithi107.stdout:1089 writesame 0x5f20402 thru 0x5f2420a data_size 0x1(0x3e09 bytes) 2024-09-21T15:33:07.923 INFO:teuthology.orchestra.run.smithi107.stdout:1090 writesame 0xaaf3d97 thru 0xaafcdff data_size 0x1(0x9069 bytes) 2024-09-21T15:33:07.928 INFO:teuthology.orchestra.run.smithi107.stdout:1091 trunc from 0xaafce00 to 0x8b00e9c 2024-09-21T15:33:07.944 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:07.944 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:07.944 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T15:33:07.945 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T15:33:07.945 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T15:33:07.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T15:33:07.950 INFO:teuthology.orchestra.run.smithi107.stdout:1093 trunc from 0x8b00e9c to 0xa580c5 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.597199055000001, 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:07.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:07.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:07.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.568987879, 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.379688510999999, 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:07.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:07.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.24830382, 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:07.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:07.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.987 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.087206416000001, 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:07.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:07.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:07.993 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.087153869, 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:07.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:07.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.087105744, 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:07.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:08.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:08.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.080851728000001, 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.005 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:08.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:08.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.074708856000001, 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.011 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:08.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.5139358739999995, 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.017 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.4469115079999995, 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.3117913770000005, 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2711505859999992, 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.271129234, 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2711160170000007, 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2711032099999997, 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2710628029999995, 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2710383350000001, 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2518163470000001, 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.1645127639999995, 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:08.071 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:08.072 INFO:teuthology.orchestra.run.smithi107.stdout:1094 read 0x647aaf thru 0x64d3ba (0x590c bytes) 2024-09-21T15:33:08.072 INFO:teuthology.orchestra.run.smithi107.stdout:1095 read 0x8cb15f thru 0x8d1b09 (0x69ab bytes) 2024-09-21T15:33:08.072 INFO:teuthology.orchestra.run.smithi107.stdout:1096 read 0x33268b thru 0x33ede3 (0xc759 bytes) 2024-09-21T15:33:08.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:1098 write 0x583f37e thru 0x58494c9 (0xa14c bytes) 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:1099 write 0x6d37003 thru 0x6d3c29a (0x5298 bytes) 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:1102 read 0x7adb44 thru 0x7ae792 (0xc4f bytes) 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:1103 trunc from 0x6d3c29b to 0xcb163f8 2024-09-21T15:33:08.074 INFO:teuthology.orchestra.run.smithi107.stdout:1104 read 0x2a5060e thru 0x2a5f4dc (0xeecf bytes) 2024-09-21T15:33:08.075 INFO:teuthology.orchestra.run.smithi107.stdout:1105 read 0x77d5b99 thru 0x77e5996 (0xfdfe bytes) 2024-09-21T15:33:08.075 INFO:teuthology.orchestra.run.smithi107.stdout:1106 writesame 0x9f77314 thru 0x9f828b6 data_size 0x1(0xb5a3 bytes) 2024-09-21T15:33:08.075 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:08.075 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:08.075 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:08.076 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:1108 trunc from 0xcb163f8 to 0xb0aa7b2 2024-09-21T15:33:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:1111 read 0x4014c14 thru 0x401f617 (0xaa04 bytes) 2024-09-21T15:33:08.076 INFO:teuthology.orchestra.run.smithi107.stdout:1112 punch from 0x98927de to 0x989b130, (0x8952 bytes) 2024-09-21T15:33:08.077 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:08.077 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:08.077 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:08.071+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:08.078 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:08.078 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T15:33:08.078 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:08.084 INFO:teuthology.orchestra.run.smithi107.stdout:1113 write 0xe778378 thru 0xe77e12c (0x5db5 bytes) 2024-09-21T15:33:08.088 INFO:teuthology.orchestra.run.smithi107.stdout:1114 punch from 0x4a338b9 to 0x4a37f02, (0x4649 bytes) 2024-09-21T15:33:08.088 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:08.088 INFO:teuthology.orchestra.run.smithi107.stdout:1117 clone 12 order 23 su 65536 sc 5 2024-09-21T15:33:08.093 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:08.093 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:08.110 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:08.110 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:08.129 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:08.129 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:08.147 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:08.147 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:08.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:08.166 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:08.173 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:08.171+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:08.195 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:08.195 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:08.209 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:08.295 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:08.295+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:08.372 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:08.372 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:08.374 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:08.371+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:08.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:08.475 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:08.475+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:08.552 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:08.552 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:08.566 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:08.576 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:08.575+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:08.677 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:08.675+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:08.731 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:08.732 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:08.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:08.778 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:08.775+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:08.879 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:08.875+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:08.908 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:08.909 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:08.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:08.980 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:08.975+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:08.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:33:09.047 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.048 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:09.107 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:09.107+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:09.109 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone11 from 0xe77e12d (overlap 0xa580c5) to 0x846704 2024-09-21T15:33:09.181 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:09.179+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:09.185 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.185 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.200 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:09.231 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T15:33:09.236 INFO:teuthology.orchestra.run.smithi107.stdout:1118 write 0x457f54a thru 0x45852a5 (0x5d5c bytes) 2024-09-21T15:33:09.283 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:09.279+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:09.285 INFO:teuthology.orchestra.run.smithi107.stdout:1119 trunc from 0xe77e12d to 0x5c95102 2024-09-21T15:33:09.337 INFO:teuthology.orchestra.run.smithi107.stdout:1120 read 0x2db6f48 thru 0x2dc5b3c (0xebf5 bytes) 2024-09-21T15:33:09.337 INFO:teuthology.orchestra.run.smithi107.stdout:1121 read 0x3b2bdb5 thru 0x3b32712 (0x695e bytes) 2024-09-21T15:33:09.338 INFO:teuthology.orchestra.run.smithi107.stdout:1122 read 0x4badd7d thru 0x4bb426a (0x64ee bytes) 2024-09-21T15:33:09.338 INFO:teuthology.orchestra.run.smithi107.stdout:1123 write 0xee66e8 thru 0xef250e (0xbe27 bytes) 2024-09-21T15:33:09.353 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:09.362 INFO:teuthology.orchestra.run.smithi107.stdout:1127 trunc from 0x5c95102 to 0xc032807 2024-09-21T15:33:09.368 INFO:teuthology.orchestra.run.smithi107.stdout:1128 read 0x9d7f7e7 thru 0x9d833d5 (0x3bef bytes) 2024-09-21T15:33:09.368 INFO:teuthology.orchestra.run.smithi107.stdout:1129 read 0xa20ae25 thru 0xa20dd74 (0x2f50 bytes) 2024-09-21T15:33:09.373 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.373 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.378 INFO:teuthology.orchestra.run.smithi107.stdout:1131 write 0xe242c21 thru 0xe2432b5 (0x695 bytes) 2024-09-21T15:33:09.383 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:09.379+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:09.383 INFO:teuthology.orchestra.run.smithi107.stdout:1132 write 0xa48f35 thru 0xa516a6 (0x8772 bytes) 2024-09-21T15:33:09.389 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:09.419 INFO:teuthology.orchestra.run.smithi107.stdout:1133 trunc from 0xe2432b6 to 0x490837d 2024-09-21T15:33:09.462 INFO:teuthology.orchestra.run.smithi107.stdout:1134 read 0x1b98d61 thru 0x1b9c3fe (0x369e bytes) 2024-09-21T15:33:09.462 INFO:teuthology.orchestra.run.smithi107.stdout:1136 read 0xd6b4f thru 0xdc962 (0x5e14 bytes) 2024-09-21T15:33:09.463 INFO:teuthology.orchestra.run.smithi107.stdout:1138 punch from 0x5ac8c8 to 0x5b7175, (0xa8ad bytes) 2024-09-21T15:33:09.479 INFO:teuthology.orchestra.run.smithi107.stdout:1139 writesame 0x9dc40ce thru 0x9dc76ac data_size 0x1(0x35df bytes) 2024-09-21T15:33:09.479 INFO:teuthology.orchestra.run.smithi107.stdout:1141 clone 13 order 22 su 2097152 sc 6 2024-09-21T15:33:09.484 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:09.479+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:09.513 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.513 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.528 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:09.584 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:09.583+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:09.650 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.650 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.664 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:09.685 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:09.683+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:09.786 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:09.783+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:09.824 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.824 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.837 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:09.886 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:09.883+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:09.962 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:09.963 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:09.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:09.979 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:33:09.987 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:09.987+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:10.088 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:10.087+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:10.104 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:10.105 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone12 from 0x9dc76ad (overlap 0x490837d) to 0x0 2024-09-21T15:33:10.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:10.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:10.189 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:10.187+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:10.290 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:10.287+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:10.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:10.317 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T15:33:10.335 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.335 INFO:teuthology.orchestra.run.smithi107.stdout:1145 read 0x1e138f5 thru 0x1e1973d (0x5e49 bytes) 2024-09-21T15:33:10.355 INFO:teuthology.orchestra.run.smithi107.stdout:1146 write 0xa0a2518 thru 0xa0aadf6 (0x88df bytes) 2024-09-21T15:33:10.357 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:10.357 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:10.359 INFO:teuthology.orchestra.run.smithi107.stdout:1147 read 0x1353234 thru 0x135b1df (0x7fac bytes) 2024-09-21T15:33:10.359 INFO:teuthology.orchestra.run.smithi107.stdout:1148 read 0x22fcd7e thru 0x2308a6c (0xbcef bytes) 2024-09-21T15:33:10.360 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.360 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.360 INFO:teuthology.orchestra.run.smithi107.stdout:1153 write 0x8c201c2 thru 0x8c26ea3 (0x6ce2 bytes) 2024-09-21T15:33:10.365 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.375 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:10.375 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:10.377 INFO:teuthology.orchestra.run.smithi107.stdout:1155 write 0xaf31334 thru 0xaf387b5 (0x7482 bytes) 2024-09-21T15:33:10.382 INFO:teuthology.orchestra.run.smithi107.stdout:1156 writesame 0x41aa9fa thru 0x41b84f6 data_size 0x3(0xdafd bytes) 2024-09-21T15:33:10.390 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:10.387+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:10.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:10.393 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:10.396 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.396 INFO:teuthology.orchestra.run.smithi107.stdout:1159 write 0x3e1a71a thru 0x3e1e9e9 (0x42d0 bytes) 2024-09-21T15:33:10.411 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:10.411 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:10.417 INFO:teuthology.orchestra.run.smithi107.stdout:1160 trunc from 0xaf387b6 to 0x1c52098 2024-09-21T15:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:10.444 INFO:teuthology.orchestra.run.smithi107.stdout:1161 read 0x5d438a thru 0x5e1535 (0xd1ac bytes) 2024-09-21T15:33:10.444 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.457 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:10.461 INFO:teuthology.orchestra.run.smithi107.stdout:1163 write 0x5de1ccf thru 0x5de71d7 (0x5509 bytes) 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:136 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'94, mlcod=15'87)", 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 38.926411696000002, 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.007421263, 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.463 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535230+0000", 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535228+0000", 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535242+0000", 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3577000000000001e-05 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535288+0000", 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6740000000000003e-05 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535307+0000", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9063e-05 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536680+0000", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013722090000000001 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.536744+0000", 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4215999999999996e-05 2024-09-21T15:33:10.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542591+0000", 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058466680000000002 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542651+0000", 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0612999999999997e-05 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4198.0:137 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'95, mlcod=15'87)", 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 38.925839619999998, 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0069096740000000002, 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535802+0000", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535801+0000", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535877+0000", 2024-09-21T15:33:10.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5865000000000006e-05 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535892+0000", 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5004000000000001e-05 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.535901+0000", 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4399999999999994e-06 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537645+0000", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00174387 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.537681+0000", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6341999999999999e-05 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542655+0000", 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049735020000000003 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:31.542712+0000", 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6727000000000001e-05 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:981 3.1 3:812dd4d6:::rbd_data.11911fd826fd.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.429356322, 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073781309999999996, 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 981 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032285+0000", 2024-09-21T15:33:10.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032284+0000", 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032291+0000", 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0099999999999998e-06 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032306+0000", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5162e-05 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032317+0000", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0641e-05 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032391+0000", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3563e-05 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032531+0000", 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014023900000000001 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032886+0000", 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035541499999999998 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035001+0000", 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0021150409999999998 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039546+0000", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045443339999999997 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039615+0000", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8936000000000005e-05 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039663+0000", 2024-09-21T15:33:10.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8866000000000003e-05 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:984 3.6 3:630b821a:::rbd_data.11911fd826fd.0000000000000033:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.428882250000001, 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006946069, 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 984 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032759+0000", 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032758+0000", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032765+0000", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4590000000000001e-06 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032780+0000", 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:33:10.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032789+0000", 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4339999999999998e-06 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032836+0000", 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6891000000000002e-05 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.032919+0000", 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3238000000000005e-05 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033108+0000", 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000188803 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035121+0000", 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020131900000000002 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044746530000000003 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039653+0000", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6486000000000001e-05 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039705+0000", 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2935000000000001e-05 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.483 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:985 3.17 3:eca73502:::rbd_data.11911fd826fd.0000000000000034:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.428472877000001, 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0094186879999999997, 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 985 2024-09-21T15:33:10.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033169+0000", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033168+0000", 2024-09-21T15:33:10.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033175+0000", 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1729999999999998e-06 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033194+0000", 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9383999999999999e-05 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033205+0000", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0947e-05 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033266+0000", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0551999999999999e-05 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033373+0000", 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107155 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033600+0000", 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022703799999999999 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035620+0000", 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020200050000000001 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042474+0000", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068545359999999996 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042537+0000", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2778000000000001e-05 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042587+0000", 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0272999999999998e-05 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.489 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:982 3.10 e27/21 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.428359688, 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063444290000000004, 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:10.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033282+0000", 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033281+0000", 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033295+0000", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3361e-05 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033306+0000", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1268e-05 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033313+0000", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6450000000000005e-06 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033365+0000", 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1208999999999997e-05 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.033391+0000", 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6353e-05 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039585+0000", 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061938990000000001 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039626+0000", 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1467999999999997e-05 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:987 3.1f 3:fa7a7055:::rbd_data.11911fd826fd.0000000000000036:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.42665276, 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0077071170000000003, 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 987 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034989+0000", 2024-09-21T15:33:10.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034988+0000", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.034996+0000", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2360000000000004e-06 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035015+0000", 2024-09-21T15:33:10.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9072e-05 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035028+0000", 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2668e-05 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035078+0000", 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0667999999999997e-05 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035162+0000", 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3707000000000003e-05 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035389+0000", 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022708 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.037465+0000", 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020759379999999998 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042600+0000", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051350750000000002 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042653+0000", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3196999999999999e-05 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042696+0000", 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2800999999999999e-05 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.499 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:988 3.6 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.426446420000001, 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073861910000000003, 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 988 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.500 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035195+0000", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035194+0000", 2024-09-21T15:33:10.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035203+0000", 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2059999999999996e-06 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035222+0000", 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9320000000000001e-05 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035233+0000", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0793e-05 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035310+0000", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6808000000000001e-05 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035479+0000", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016872999999999999 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:10.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035835+0000", 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035607999999999999 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039777+0000", 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039426979999999997 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042513+0000", 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002735786 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042550+0000", 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7211e-05 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.042581+0000", 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1031000000000002e-05 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3169 3.b e30/21 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head v 30'28, mlcod=29'25)", 2024-09-21T15:33:10.505 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.576866576, 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010485235000000001, 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884775+0000", 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884773+0000", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884789+0000", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.556e-05 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884822+0000", 2024-09-21T15:33:10.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3189999999999999e-05 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884860+0000", 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8166e-05 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884989+0000", 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012823500000000001 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885053+0000", 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4503000000000002e-05 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895205+0000", 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010151415 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895260+0000", 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5736000000000003e-05 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3170 3.17 3:eb82c48b:::rbd_data.1191ab8bc3c6.0000000000000001:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.574631220000001, 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0095184950000000001, 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3170 2024-09-21T15:33:10.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887010+0000", 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887007+0000", 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887024+0000", 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.647e-05 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887072+0000", 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8038999999999999e-05 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887096+0000", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3777e-05 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887229+0000", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133805 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887459+0000", 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000229535 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.887904+0000", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044505 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892712+0000", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048083960000000004 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896348+0000", 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036350990000000001 2024-09-21T15:33:10.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896471+0000", 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012388300000000001 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.896529+0000", 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7491000000000002e-05 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.515 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3172 3.b e30/21 3:d0ec678c:::rbd_data.1191ab8bc3c6.0000000000000003:head v 30'30, mlcod=29'25)", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.569579837999999, 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073246989999999996, 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892062+0000", 2024-09-21T15:33:10.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892061+0000", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892072+0000", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1301000000000001e-05 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892087+0000", 2024-09-21T15:33:10.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.504e-05 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892097+0000", 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9969999999999993e-06 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892204+0000", 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106783 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892258+0000", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4014000000000002e-05 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899269+0000", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070109120000000002 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899386+0000", 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117793 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.519 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3173 3.1 3:84db22e7:::rbd_data.1191ab8bc3c6.0000000000000004:head [stat,truncate 6032748] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.569369353999999, 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0071419710000000004, 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3173 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892272+0000", 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892269+0000", 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892286+0000", 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6826000000000001e-05 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892323+0000", 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7292999999999997e-05 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892345+0000", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2186000000000001e-05 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892469+0000", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000123566 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.892666+0000", 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019718900000000001 2024-09-21T15:33:10.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893107+0000", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00044130900000000003 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895184+0000", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020761920000000001 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899255+0000", 2024-09-21T15:33:10.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040709980000000002 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899355+0000", 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100543 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.899414+0000", 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9119000000000001e-05 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.751974138, 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:10.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:10.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:10.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.741651843, 2024-09-21T15:33:10.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:10.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.8141546179999999, 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.536 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:10.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:10.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7802431839999997, 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:10.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5749 3.6 3:65672256:::rbd_header.11917ba4f9b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.5476413490000001, 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063958840000000001, 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5749 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914000+0000", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.913998+0000", 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914014+0000", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6646e-05 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914047+0000", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2270000000000001e-05 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914070+0000", 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3572999999999999e-05 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914205+0000", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013493899999999999 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.914476+0000", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027055800000000001 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.916280+0000", 2024-09-21T15:33:10.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018040560000000001 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.919876+0000", 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035958740000000002 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920299+0000", 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042360999999999999 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920352+0000", 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.253e-05 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.920396+0000", 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4418000000000001e-05 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7841 3.1 3:8073bacf:::rbd_object_map.1191ebf845d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.951608+0000", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.5100335669999998, 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0063147289999999998, 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7841 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951608+0000", 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951608+0000", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951608+0000", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951611+0000", 2024-09-21T15:33:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5319999999999998e-06 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951617+0000", 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.626e-06 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951624+0000", 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6719999999999998e-06 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.953700+0000", 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020765190000000002 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.953795+0000", 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4840000000000007e-05 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.954160+0000", 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036508999999999999 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.955492+0000", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013321260000000001 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.957810+0000", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0023179379999999999 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.957876+0000", 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6068999999999994e-05 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.957923+0000", 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6631999999999997e-05 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7842 3.1 3:8073bacf:::rbd_object_map.1191ebf845d7:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.951656+0000", 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.5099857290000003, 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0072043979999999999, 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7842 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951656+0000", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951656+0000", 2024-09-21T15:33:10.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951655+0000", 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951663+0000", 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8200000000000003e-06 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951680+0000", 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6133999999999998e-05 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.951691+0000", 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1148000000000001e-05 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.954603+0000", 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029122010000000001 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.954656+0000", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2868999999999998e-05 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.954859+0000", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020360699999999999 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.958500+0000", 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036407980000000002 2024-09-21T15:33:10.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.958753+0000", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00025284599999999998 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.958804+0000", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0858e-05 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.958860+0000", 2024-09-21T15:33:10.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6372000000000002e-05 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:9697 3.18 3:1828185b:::rbd_data.1191f1341b63.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:10.386657+0000", 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.074984797000000006, 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.010322975, 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:10.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 9697 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386657+0000", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386657+0000", 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:10.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386656+0000", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386662+0000", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7610000000000002e-06 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386677+0000", 2024-09-21T15:33:10.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5155e-05 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386688+0000", 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0312999999999999e-05 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386730+0000", 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2225999999999999e-05 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:10.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.386824+0000", 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4166999999999995e-05 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.388565+0000", 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0017403550000000001 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.396876+0000", 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0083113400000000004 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.396905+0000", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9624999999999998e-05 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.396941+0000", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5021999999999998e-05 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.396980+0000", 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9283000000000003e-05 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:10.569 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:10.569 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:10.570 INFO:teuthology.orchestra.run.smithi107.stdout:1165 read 0x453f0fd thru 0x454d288 (0xe18c bytes) 2024-09-21T15:33:10.570 INFO:teuthology.orchestra.run.smithi107.stdout:1166 punch from 0x3d55213 to 0x3d623e6, (0xd1d3 bytes) 2024-09-21T15:33:10.570 INFO:teuthology.orchestra.run.smithi107.stdout:1167 read 0x5241b9e thru 0x52441fd (0x2660 bytes) 2024-09-21T15:33:10.570 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:10.570 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:10.571 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:10.572 INFO:teuthology.orchestra.run.smithi107.stdout:1168 writesame 0xa62f2c0 thru 0xa63153e data_size 0x1(0x227f bytes) 2024-09-21T15:33:10.572 INFO:teuthology.orchestra.run.smithi107.stdout:1170 trunc from 0xa63153f to 0x79c440 2024-09-21T15:33:10.572 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:10.573 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:10.573 INFO:teuthology.orchestra.run.smithi107.stdout:1171 write 0xe17a9e4 thru 0xe1879b8 (0xcfd5 bytes) 2024-09-21T15:33:10.573 INFO:teuthology.orchestra.run.smithi107.stdout:1172 write 0x799ae0e thru 0x79a973b (0xe92e bytes) 2024-09-21T15:33:10.574 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.574 INFO:teuthology.orchestra.run.smithi107.stdout:1175 read 0x11bed8f thru 0x11c8e49 (0xa0bb bytes) 2024-09-21T15:33:10.574 INFO:teuthology.orchestra.run.smithi107.stdout:1176 read 0x914ed8c thru 0x9150609 (0x187e bytes) 2024-09-21T15:33:10.574 INFO:teuthology.orchestra.run.smithi107.stdout:1177 write 0xbea8aa3 thru 0xbeb7b2d (0xf08b bytes) 2024-09-21T15:33:10.574 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:10.567+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:10.575 INFO:teuthology.orchestra.run.smithi107.stdout:1178 punch from 0x6b917c3 to 0x6b9965c, (0x7e99 bytes) 2024-09-21T15:33:10.575 INFO:teuthology.orchestra.run.smithi107.stdout:1180 write 0x1ebe445 thru 0x1ec1493 (0x304f bytes) 2024-09-21T15:33:10.579 INFO:teuthology.orchestra.run.smithi107.stdout:1182 punch from 0x3561f0f to 0x35703c0, (0xe4b1 bytes) 2024-09-21T15:33:10.589 INFO:teuthology.orchestra.run.smithi107.stdout:1183 trunc from 0xe1879b9 to 0x91fb6f5 2024-09-21T15:33:10.606 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.619 INFO:teuthology.orchestra.run.smithi107.stdout:1185 write 0xb0ec67f thru 0xb0fb135 (0xeab7 bytes) 2024-09-21T15:33:10.624 INFO:teuthology.orchestra.run.smithi107.stdout:1187 read 0xe3fd82 thru 0xe49873 (0x9af2 bytes) 2024-09-21T15:33:10.625 INFO:teuthology.orchestra.run.smithi107.stdout:1188 read 0x34916dc thru 0x349d1a5 (0xbaca bytes) 2024-09-21T15:33:10.625 INFO:teuthology.orchestra.run.smithi107.stdout:1189 writesame 0x33be50f thru 0x33bf901 data_size 0x1(0x13f3 bytes) 2024-09-21T15:33:10.633 INFO:teuthology.orchestra.run.smithi107.stdout:1191 write 0x9f45ea3 thru 0x9f4e835 (0x8993 bytes) 2024-09-21T15:33:10.638 INFO:teuthology.orchestra.run.smithi107.stdout:1192 read 0x1ceda73 thru 0x1cfc714 (0xeca2 bytes) 2024-09-21T15:33:10.638 INFO:teuthology.orchestra.run.smithi107.stdout:1194 read 0x2085442 thru 0x208dc0f (0x87ce bytes) 2024-09-21T15:33:10.638 INFO:teuthology.orchestra.run.smithi107.stdout:1195 read 0xcf170e thru 0xcf9aa7 (0x839a bytes) 2024-09-21T15:33:10.639 INFO:teuthology.orchestra.run.smithi107.stdout:1197 writesame 0x44d086a thru 0x44d2b5f data_size 0x19(0x22f6 bytes) 2024-09-21T15:33:10.642 INFO:teuthology.orchestra.run.smithi107.stdout:1198 write 0x7cfcdbe thru 0x7cff45e (0x26a1 bytes) 2024-09-21T15:33:10.647 INFO:teuthology.orchestra.run.smithi107.stdout:1199 write 0x6ffed31 thru 0x700a712 (0xb9e2 bytes) 2024-09-21T15:33:10.653 INFO:teuthology.orchestra.run.smithi107.stdout:1201 trunc from 0xb0fb136 to 0xb3e8655 2024-09-21T15:33:10.664 INFO:teuthology.orchestra.run.smithi107.stdout:1202 read 0x4543eec thru 0x454c320 (0x8435 bytes) 2024-09-21T15:33:10.670 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:10.667+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:10.679 INFO:teuthology.orchestra.run.smithi107.stdout:1203 writesame 0xc49b925 thru 0xc4a48a8 data_size 0x2c(0x8f84 bytes) 2024-09-21T15:33:10.679 INFO:teuthology.orchestra.run.smithi107.stdout:1204 write 0x25cae59 thru 0x25cb6ab (0x853 bytes) 2024-09-21T15:33:10.685 INFO:teuthology.orchestra.run.smithi107.stdout:1205 writesame 0x7277490 thru 0x7285444 data_size 0x1(0xdfb5 bytes) 2024-09-21T15:33:10.702 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:10.702 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:10.702 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:10.703 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:10.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:10.762 INFO:teuthology.orchestra.run.smithi107.stdout:1206 write 0x4cb1e10 thru 0x4cc0748 (0xe939 bytes) 2024-09-21T15:33:10.784 INFO:teuthology.orchestra.run.smithi107.stdout:1208 trunc from 0xc4a48a9 to 0x5d636c3 2024-09-21T15:33:10.785 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:10.783+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:10.804 INFO:teuthology.orchestra.run.smithi107.stdout:1209 read 0x5727303 thru 0x572aee3 (0x3be1 bytes) 2024-09-21T15:33:10.819 INFO:teuthology.orchestra.run.smithi107.stdout:1210 trunc from 0x5d636c3 to 0xc253260 2024-09-21T15:33:10.830 INFO:teuthology.orchestra.run.smithi107.stdout:1211 write 0xad4c465 thru 0xad582b5 (0xbe51 bytes) 2024-09-21T15:33:10.836 INFO:teuthology.orchestra.run.smithi107.stdout:1213 writesame 0x21e5965 thru 0x21efd59 data_size 0x11(0xa3f5 bytes) 2024-09-21T15:33:10.867 INFO:teuthology.orchestra.run.smithi107.stdout:1215 trunc from 0xc253260 to 0x1ce0d2c 2024-09-21T15:33:10.872 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:10.871+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:10.896 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:10.897 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:10.915 INFO:teuthology.orchestra.run.smithi107.stdout:1216 trunc from 0x1ce0d2c to 0x9ee7dbc 2024-09-21T15:33:10.923 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:10.928 INFO:teuthology.orchestra.run.smithi107.stdout:1217 read 0x95e4836 thru 0x95e899f (0x416a bytes) 2024-09-21T15:33:10.928 INFO:teuthology.orchestra.run.smithi107.stdout:1218 punch from 0x72f74a5 to 0x7303cbd, (0xc818 bytes) 2024-09-21T15:33:10.943 INFO:teuthology.orchestra.run.smithi107.stdout:1219 writesame 0xc735862 thru 0xc735995 data_size 0x2(0x134 bytes) 2024-09-21T15:33:10.946 INFO:teuthology.orchestra.run.smithi107.stdout:1220 trunc from 0xc735996 to 0xe6b1f6e 2024-09-21T15:33:10.958 INFO:teuthology.orchestra.run.smithi107.stdout:1223 trunc from 0xe6b1f6e to 0xd7d5679 2024-09-21T15:33:10.971 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:10.971 INFO:teuthology.orchestra.run.smithi107.stdout:1225 read 0x44e1004 thru 0x44ed1bb (0xc1b8 bytes) 2024-09-21T15:33:10.971 INFO:teuthology.orchestra.run.smithi107.stdout:1227 writesame 0x1569dd5 thru 0x1575ebd data_size 0x55(0xc0e9 bytes) 2024-09-21T15:33:10.972 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:10.971+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:10.978 INFO:teuthology.orchestra.run.smithi107.stdout:1228 read 0x825cd46 thru 0x825f657 (0x2912 bytes) 2024-09-21T15:33:10.978 INFO:teuthology.orchestra.run.smithi107.stdout:1230 writesame 0xb3ef55 thru 0xb458d6 data_size 0x4a(0x6982 bytes) 2024-09-21T15:33:10.983 INFO:teuthology.orchestra.run.smithi107.stdout:1231 write 0x9e27954 thru 0x9e33fa6 (0xc653 bytes) 2024-09-21T15:33:10.988 INFO:teuthology.orchestra.run.smithi107.stdout:1232 read 0xa1eb1e thru 0xa20dcb (0x22ae bytes) 2024-09-21T15:33:10.989 INFO:teuthology.orchestra.run.smithi107.stdout:1233 read 0xa39378e thru 0xa39c227 (0x8a9a bytes) 2024-09-21T15:33:10.989 INFO:teuthology.orchestra.run.smithi107.stdout:1235 writesame 0x4e2a62f thru 0x4e34311 data_size 0x1(0x9ce3 bytes) 2024-09-21T15:33:11.047 INFO:teuthology.orchestra.run.smithi107.stdout:1236 write 0xc57b807 thru 0xc587e76 (0xc670 bytes) 2024-09-21T15:33:11.054 INFO:teuthology.orchestra.run.smithi107.stdout:1237 write 0x3745892 thru 0x374a3fc (0x4b6b bytes) 2024-09-21T15:33:11.058 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:11.058 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:11.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:11.058 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:11.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.323589828999999, 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:33:11.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.513178206999999, 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.063 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:11.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:33:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.477567057999998, 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:33:11.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.477486623000001, 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:33:11.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:33:11.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:33:11.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.477441478999999, 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:33:11.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.082 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.477263251, 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:11.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.477189600999999, 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.087 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:33:11.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.059182774, 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:33:11.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:33:11.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "pg_update_log_missing(3.8 epoch 27/21 rep_tid 41 entries 27'3 (0'0) error 3:15312f39:::rbd_data.11911fd826fd.0000000000000020:head by client.4497.0:950 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.038193225000001, 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037814699999999999, 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017199+0000", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017205+0000", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7449999999999998e-06 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017223+0000", 2024-09-21T15:33:11.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7626999999999999e-05 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017241+0000", 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.749e-05 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.017302+0000", 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1802000000000002e-05 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.020980+0000", 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036779909999999998 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:988 3.6 e27/21 3:6539e120:::rbd_data.11911fd826fd.000000000000002f:head v 27'10, mlcod=26'6)", 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.019436116000001, 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036395920000000001, 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035956+0000", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035955+0000", 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035964+0000", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3239999999999992e-06 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035980+0000", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5855e-05 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.035988+0000", 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8650000000000001e-06 2024-09-21T15:33:11.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036064+0000", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6191000000000003e-05 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.036088+0000", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3470000000000001e-05 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039519+0000", 2024-09-21T15:33:11.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034318809999999999 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.039596+0000", 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6165e-05 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5584 3.1a e33/21 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head v 33'40, mlcod=32'38)", 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:11.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.0031826959999997, 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053598999999999999, 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052208+0000", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056205+0000", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003997202 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056221+0000", 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6167e-05 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056231+0000", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.55e-06 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056297+0000", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6231999999999999e-05 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056329+0000", 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1977999999999998e-05 2024-09-21T15:33:11.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057476+0000", 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011471280000000001 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057569+0000", 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3178999999999997e-05 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7071 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.1506347269999999, 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041211169999999997, 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7071 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904762+0000", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904770+0000", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5879999999999998e-06 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904777+0000", 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6170000000000003e-06 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905465+0000", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00068793299999999995 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905513+0000", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7842e-05 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905832+0000", 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031906099999999999 2024-09-21T15:33:11.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908509+0000", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026772739999999999 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908784+0000", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027538999999999998 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908833+0000", 2024-09-21T15:33:11.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8958000000000001e-05 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908879+0000", 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5105999999999998e-05 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7075 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.1445664750000004, 2024-09-21T15:33:11.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045190079999999997, 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7075 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910825+0000", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910830+0000", 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:33:11.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910837+0000", 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2740000000000002e-06 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910842+0000", 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3789999999999997e-06 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911544+0000", 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00070163500000000004 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911629+0000", 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4837000000000001e-05 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912033+0000", 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040393900000000002 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915180+0000", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031479189999999999 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915295+0000", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114585 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915324+0000", 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8923000000000001e-05 2024-09-21T15:33:11.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915345+0000", 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0659e-05 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7320 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095534+0000", 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.9598585050000001, 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036848620000000001, 2024-09-21T15:33:11.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7320 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095534+0000", 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095534+0000", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095533+0000", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095539+0000", 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.331e-06 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095553+0000", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3981999999999999e-05 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095559+0000", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6550000000000001e-06 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095610+0000", 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1688999999999999e-05 2024-09-21T15:33:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095698+0000", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7780000000000003e-05 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096032+0000", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033443299999999997 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.099140+0000", 2024-09-21T15:33:11.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031071969999999999 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.099147+0000", 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.029e-06 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.099189+0000", 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2561999999999997e-05 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.099218+0000", 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9206000000000001e-05 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7321 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.95981352, 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053282879999999996, 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7321 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095578+0000", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095581+0000", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2490000000000001e-06 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095588+0000", 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4489999999999996e-06 2024-09-21T15:33:11.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095591+0000", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096412+0000", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00082109700000000002 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096489+0000", 2024-09-21T15:33:11.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6391999999999996e-05 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096833+0000", 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034386699999999998 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100563+0000", 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037307350000000002 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100810+0000", 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024649300000000003 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100870+0000", 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0096000000000002e-05 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100907+0000", 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7072e-05 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7322 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.959779685, 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075079489999999999, 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7322 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095614+0000", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095619+0000", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.904e-06 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095622+0000", 2024-09-21T15:33:11.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.847e-06 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097310+0000", 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016879639999999999 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097392+0000", 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1959999999999995e-05 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097739+0000", 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034728500000000002 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102987+0000", 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052481949999999998 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103053+0000", 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5667000000000007e-05 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103082+0000", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8691999999999999e-05 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103120+0000", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8568000000000001e-05 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7323 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:11.136 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.9597617789999999, 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076219080000000002, 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7323 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095632+0000", 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.074e-06 2024-09-21T15:33:11.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095641+0000", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0920000000000004e-06 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095649+0000", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.174e-06 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098290+0000", 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002641071 2024-09-21T15:33:11.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098373+0000", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2400999999999999e-05 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098712+0000", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033931599999999999 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103134+0000", 2024-09-21T15:33:11.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044221629999999998 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103176+0000", 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1721999999999998e-05 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103221+0000", 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5046000000000002e-05 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103252+0000", 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1241999999999997e-05 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7329 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.107465+0000", 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.9479271040000001, 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038599020000000001, 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7329 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107465+0000", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107465+0000", 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107464+0000", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107469+0000", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8539999999999996e-06 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107479+0000", 2024-09-21T15:33:11.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0086e-05 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107484+0000", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0860000000000001e-06 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107538+0000", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3823000000000003e-05 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107625+0000", 2024-09-21T15:33:11.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6502000000000006e-05 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.107969+0000", 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034394699999999999 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109616+0000", 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016470969999999999 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111259+0000", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001643254 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111300+0000", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1124999999999997e-05 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111325+0000", 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4919999999999999e-05 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.147 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7330 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.9473165740000002, 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057881219999999997, 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7330 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.148 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108075+0000", 2024-09-21T15:33:11.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108079+0000", 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5979999999999999e-06 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108087+0000", 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108097+0000", 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0287e-05 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108474+0000", 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000376635 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108566+0000", 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1529999999999999e-05 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108932+0000", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036658600000000002 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111203+0000", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022704840000000001 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113816+0000", 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026135889999999999 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113846+0000", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9471999999999998e-05 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113864+0000", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7903000000000001e-05 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7812 3.19 3:9b58d28c:::rbd_data.1191ebf845d7.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:11.153 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.1499290690000001, 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0085838960000000006, 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7812 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905462+0000", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905467+0000", 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2769999999999996e-06 2024-09-21T15:33:11.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905478+0000", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0740999999999999e-05 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905483+0000", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0300000000000001e-06 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905535+0000", 2024-09-21T15:33:11.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1724000000000001e-05 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905597+0000", 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1644999999999997e-05 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.907085+0000", 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001488917 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:11.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.913915+0000", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068291209999999996 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.913967+0000", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1981999999999998e-05 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.914002+0000", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5743e-05 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.914047+0000", 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4613999999999997e-05 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.159 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:11.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1239 write 0xeb5ee96 thru 0xeb68529 (0x9694 bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1240 write 0x94b1e6e thru 0x94bb699 (0x982c bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1241 read 0x7c2da95 thru 0x7c2e541 (0xaad bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1242 read 0x18d538b thru 0x18d9cc1 (0x4937 bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1244 read 0xc773368 thru 0xc77b042 (0x7cdb bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1246 read 0xb0faede thru 0xb104413 (0x9536 bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1248 write 0xddf7860 thru 0xddfd696 (0x5e37 bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1250 writesame 0x7c32a5a thru 0x7c3defd data_size 0x16(0xb4a4 bytes) 2024-09-21T15:33:11.162 INFO:teuthology.orchestra.run.smithi107.stdout:1251 read 0x57bbc18 thru 0x57c0418 (0x4801 bytes) 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1255 punch from 0x8e0cfb8 to 0x8e0eb59, (0x1ba1 bytes) 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1257 write 0xc9126c9 thru 0xc9159ea (0x3322 bytes) 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1259 read 0x31055e0 thru 0x3112dae (0xd7cf bytes) 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1260 punch from 0xbd850a3 to 0xbd876f8, (0x2655 bytes) 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1261 trunc from 0xeb6852a to 0xce4137d 2024-09-21T15:33:11.163 INFO:teuthology.orchestra.run.smithi107.stdout:1262 trunc from 0xce4137d to 0x88032ef 2024-09-21T15:33:11.163 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:11.159+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:11.179 INFO:teuthology.orchestra.run.smithi107.stdout:1264 write 0xcaf09b0 thru 0xcaf0e3a (0x48b bytes) 2024-09-21T15:33:11.184 INFO:teuthology.orchestra.run.smithi107.stdout:1266 writesame 0x684c822 thru 0x685c70b data_size 0x43(0xfeea bytes) 2024-09-21T15:33:11.201 INFO:teuthology.orchestra.run.smithi107.stdout:1267 write 0xcb37003 thru 0xcb3fc97 (0x8c95 bytes) 2024-09-21T15:33:11.203 INFO:teuthology.orchestra.run.smithi107.stdout:1268 punch from 0x95bbc0e to 0x95be29d, (0x268f bytes) 2024-09-21T15:33:11.220 INFO:teuthology.orchestra.run.smithi107.stdout:1269 trunc from 0xcb3fc98 to 0x320dda1 2024-09-21T15:33:11.261 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:11.259+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:11.272 INFO:teuthology.orchestra.run.smithi107.stdout:1270 write 0xe2e2122 thru 0xe2f0bd5 (0xeab4 bytes) 2024-09-21T15:33:11.277 INFO:teuthology.orchestra.run.smithi107.stdout:1272 punch from 0x98da769 to 0x98e6b3a, (0xc3d1 bytes) 2024-09-21T15:33:11.289 INFO:teuthology.orchestra.run.smithi107.stdout:1273 write 0xea388b7 thru 0xea3c0cc (0x3816 bytes) 2024-09-21T15:33:11.293 INFO:teuthology.orchestra.run.smithi107.stdout:1275 punch from 0x42f8d2e to 0x42fe41c, (0x56ee bytes) 2024-09-21T15:33:11.293 INFO:teuthology.orchestra.run.smithi107.stdout:1276 write 0x28c4964 thru 0x28ca92f (0x5fcc bytes) 2024-09-21T15:33:11.297 INFO:teuthology.orchestra.run.smithi107.stdout:1277 write 0x2761d82 thru 0x2769bda (0x7e59 bytes) 2024-09-21T15:33:11.303 INFO:teuthology.orchestra.run.smithi107.stdout:1278 write 0x6458730 thru 0x645efbc (0x688d bytes) 2024-09-21T15:33:11.307 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.307 INFO:teuthology.orchestra.run.smithi107.stdout:1282 trunc from 0xea3c0cd to 0xec62dfc 2024-09-21T15:33:11.315 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:11.315 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:11.315 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:11.315 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:11.318 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.318 INFO:teuthology.orchestra.run.smithi107.stdout:1284 read 0xe8f72b5 thru 0xe8fc443 (0x518f bytes) 2024-09-21T15:33:11.318 INFO:teuthology.orchestra.run.smithi107.stdout:1285 write 0x2b25624 thru 0x2b3466e (0xf04b bytes) 2024-09-21T15:33:11.325 INFO:teuthology.orchestra.run.smithi107.stdout:1286 punch from 0xbe8d481 to 0xbe99058, (0xbbd7 bytes) 2024-09-21T15:33:11.331 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:11.340 INFO:teuthology.orchestra.run.smithi107.stdout:1287 trunc from 0xec62dfc to 0x7cbb6b5 2024-09-21T15:33:11.361 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:11.359+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:11.376 INFO:teuthology.orchestra.run.smithi107.stdout:1291 write 0xbe6032d thru 0xbe69b19 (0x97ed bytes) 2024-09-21T15:33:11.381 INFO:teuthology.orchestra.run.smithi107.stdout:1292 read 0xb41cc35 thru 0xb41e872 (0x1c3e bytes) 2024-09-21T15:33:11.381 INFO:teuthology.orchestra.run.smithi107.stdout:1293 read 0x7b94e16 thru 0x7b9ac43 (0x5e2e bytes) 2024-09-21T15:33:11.389 INFO:teuthology.orchestra.run.smithi107.stdout:1294 trunc from 0xbe69b1a to 0x803efcd 2024-09-21T15:33:11.407 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.407 INFO:teuthology.orchestra.run.smithi107.stdout:1296 punch from 0x78e5fc0 to 0x78efd3d, (0x9d7d bytes) 2024-09-21T15:33:11.407 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1298 trunc from 0x803efcd to 0x226f3ed 2024-09-21T15:33:11.444 INFO:teuthology.orchestra.run.smithi107.stdout:1299 writesame 0x1cd2c63 thru 0x1ce081e data_size 0xa4(0xdbbc bytes) 2024-09-21T15:33:11.447 INFO:teuthology.orchestra.run.smithi107.stdout:1300 punch from 0x13590c2 to 0x135ee23, (0x5d61 bytes) 2024-09-21T15:33:11.462 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:11.459+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:11.465 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:11.465 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:11.465 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:11.465 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:11.465 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:11.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:11.482 INFO:teuthology.orchestra.run.smithi107.stdout:1301 writesame 0xc1255df thru 0xc12ebf5 data_size 0xb(0x9617 bytes) 2024-09-21T15:33:11.483 INFO:teuthology.orchestra.run.smithi107.stdout:1303 writesame 0x4449f01 thru 0x4450598 data_size 0x43(0x6698 bytes) 2024-09-21T15:33:11.487 INFO:teuthology.orchestra.run.smithi107.stdout:1304 read 0x85d345 thru 0x862af9 (0x57b5 bytes) 2024-09-21T15:33:11.489 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.506 INFO:teuthology.orchestra.run.smithi107.stdout:1306 trunc from 0xc12ebf6 to 0x48a1e84 2024-09-21T15:33:11.517 INFO:teuthology.orchestra.run.smithi107.stdout:1307 punch from 0x449bacc to 0x44a9676, (0xdbaa bytes) 2024-09-21T15:33:11.517 INFO:teuthology.orchestra.run.smithi107.stdout:1308 punch from 0x31f65f0 to 0x31fd32d, (0x6d3d bytes) 2024-09-21T15:33:11.539 INFO:teuthology.orchestra.run.smithi107.stdout:1310 write 0x8e06f1a thru 0x8e0b228 (0x430f bytes) 2024-09-21T15:33:11.564 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:11.563+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:11.564 INFO:teuthology.orchestra.run.smithi107.stdout:1312 write 0xd2e1e08 thru 0xd2efa51 (0xdc4a bytes) 2024-09-21T15:33:11.583 INFO:teuthology.orchestra.run.smithi107.stdout:1314 write 0xdac7b22 thru 0xdacd9de (0x5ebd bytes) 2024-09-21T15:33:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:1315 read 0xa51b799 thru 0xa522ec7 (0x772f bytes) 2024-09-21T15:33:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:1317 write 0x546aaf0 thru 0x5473ba7 (0x90b8 bytes) 2024-09-21T15:33:11.597 INFO:teuthology.orchestra.run.smithi107.stdout:1320 write 0x5ff3d4 thru 0x60b97a (0xc5a7 bytes) 2024-09-21T15:33:11.600 INFO:teuthology.orchestra.run.smithi107.stdout:1323 punch from 0x7b418fc to 0x7b42de4, (0x14e8 bytes) 2024-09-21T15:33:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:1324 trunc from 0xdacd9df to 0x5a898ab 2024-09-21T15:33:11.627 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:11.627 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:11.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:11.627 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:641 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'8, mlcod=26'6)", 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.148986417, 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001951022, 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474657+0000", 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474747+0000", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9559000000000001e-05 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474777+0000", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.991e-05 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474790+0000", 2024-09-21T15:33:11.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3246e-05 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474904+0000", 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011387600000000001 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474973+0000", 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8847e-05 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476526+0000", 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015536180000000001 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476610+0000", 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3931999999999995e-05 2024-09-21T15:33:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.079704080999999, 2024-09-21T15:33:11.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:33:11.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3130 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'17, mlcod=28'14)", 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.799968963, 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018694930000000001, 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:11.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823676+0000", 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823685+0000", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823711+0000", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6106e-05 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823726+0000", 2024-09-21T15:33:11.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4644000000000001e-05 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823829+0000", 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102505 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823867+0000", 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8606999999999998e-05 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825433+0000", 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015654810000000001 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825546+0000", 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113416 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.642 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.608787810000001, 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:11.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:33:11.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:33:11.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:33:11.651 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.598997718, 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:33:11.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:33:11.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.660 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.585043678, 2024-09-21T15:33:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:11.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:33:11.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:33:11.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.576876533, 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:33:11.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:33:11.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.576130134, 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:33:11.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.557739158, 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:33:11.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:33:11.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.496873217999999, 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:33:11.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.485202881999999, 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:33:11.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:33:11.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7045 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'170, mlcod=35'167)", 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.7566575570000005, 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001819846, 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.693 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866988+0000", 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866987+0000", 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.866995+0000", 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0930000000000003e-06 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867006+0000", 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0781999999999999e-05 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867035+0000", 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9026e-05 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867077+0000", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1337000000000001e-05 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.867099+0000", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2486000000000001e-05 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868767+0000", 2024-09-21T15:33:11.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00166782 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.868808+0000", 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1032000000000003e-05 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7071 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'176, mlcod=35'173)", 2024-09-21T15:33:11.698 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.7177349509999997, 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027688610000000001, 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905910+0000", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905929+0000", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9409999999999999e-05 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905943+0000", 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4109e-05 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905950+0000", 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905995+0000", 2024-09-21T15:33:11.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4925000000000001e-05 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.906020+0000", 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4771000000000001e-05 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908648+0000", 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002627622 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908680+0000", 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1795999999999998e-05 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7075 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'178, mlcod=35'175)", 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.7115183519999997, 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030847729999999999, 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.704 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912134+0000", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5449999999999998e-06 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912146+0000", 2024-09-21T15:33:11.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1834999999999999e-05 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912153+0000", 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7819999999999996e-06 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912186+0000", 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3667999999999997e-05 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912199+0000", 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2835e-05 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915184+0000", 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029846120000000002 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915212+0000", 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8242000000000002e-05 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5267277840000002, 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:33:11.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:33:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:33:11.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5258169280000002, 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:33:11.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.524851001, 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:33:11.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:33:11.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.721 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7329 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'206, mlcod=36'204)", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5155775800000004, 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003080213, 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108067+0000", 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108077+0000", 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7990000000000005e-06 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108098+0000", 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1523999999999999e-05 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108109+0000", 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0567999999999999e-05 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108153+0000", 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4187999999999998e-05 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108175+0000", 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2089000000000001e-05 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111116+0000", 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029405490000000002 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111148+0000", 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2619000000000001e-05 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.726 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5146470189999999, 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9864 3.9 e38/21 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head v 38'41, mlcod=37'39)", 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 1.108182453, 2024-09-21T15:33:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026427170000000002, 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515462+0000", 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516535+0000", 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001073084 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516551+0000", 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5321000000000001e-05 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516560+0000", 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8259999999999999e-06 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516610+0000", 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0844999999999999e-05 2024-09-21T15:33:11.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516625+0000", 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518072+0000", 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001446578 2024-09-21T15:33:11.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518106+0000", 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4164e-05 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:11.738 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:11.739 INFO:teuthology.orchestra.run.smithi107.stdout:1325 punch from 0x1604e34 to 0x1606ca8, (0x1e74 bytes) 2024-09-21T15:33:11.740 INFO:teuthology.orchestra.run.smithi107.stdout:1326 read 0x29f6dcc thru 0x2a0624f (0xf484 bytes) 2024-09-21T15:33:11.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:11.741 INFO:teuthology.orchestra.run.smithi107.stdout:1328 trunc from 0x5a898ab to 0xc9c1985 2024-09-21T15:33:11.741 INFO:teuthology.orchestra.run.smithi107.stdout:1330 read 0x9b74876 thru 0x9b75cb6 (0x1441 bytes) 2024-09-21T15:33:11.741 INFO:teuthology.orchestra.run.smithi107.stdout:1331 writesame 0x56a0565 thru 0x56aae41 data_size 0x1(0xa8dd bytes) 2024-09-21T15:33:11.741 INFO:teuthology.orchestra.run.smithi107.stdout:1332 writesame 0xac89232 thru 0xac8a7ae data_size 0x1(0x157d bytes) 2024-09-21T15:33:11.742 INFO:teuthology.orchestra.run.smithi107.stdout:1334 read 0xae35456 thru 0xae42b41 (0xd6ec bytes) 2024-09-21T15:33:11.742 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:11.739+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:11.758 INFO:teuthology.orchestra.run.smithi107.stdout:1337 trunc from 0xc9c1985 to 0x427bcb5 2024-09-21T15:33:11.784 INFO:teuthology.orchestra.run.smithi107.stdout:1340 punch from 0x190325a to 0x1907954, (0x46fa bytes) 2024-09-21T15:33:11.784 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.784 INFO:teuthology.orchestra.run.smithi107.stdout:1342 read 0x29bec4a thru 0x29ce7f0 (0xfba7 bytes) 2024-09-21T15:33:11.784 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.792 INFO:teuthology.orchestra.run.smithi107.stdout:1344 trunc from 0x427bcb5 to 0x70671b8 2024-09-21T15:33:11.806 INFO:teuthology.orchestra.run.smithi107.stdout:1345 punch from 0x17cea27 to 0x17da053, (0xb62c bytes) 2024-09-21T15:33:11.806 INFO:teuthology.orchestra.run.smithi107.stdout:1346 read 0x6c7b721 thru 0x6c87caa (0xc58a bytes) 2024-09-21T15:33:11.806 INFO:teuthology.orchestra.run.smithi107.stdout:1349 read 0x5b06381 thru 0x5b0d85e (0x74de bytes) 2024-09-21T15:33:11.806 INFO:teuthology.orchestra.run.smithi107.stdout:1350 read 0x600d2e0 thru 0x601243b (0x515c bytes) 2024-09-21T15:33:11.813 INFO:teuthology.orchestra.run.smithi107.stdout:1351 trunc from 0x70671b8 to 0xa3de90c 2024-09-21T15:33:11.827 INFO:teuthology.orchestra.run.smithi107.stdout:1353 read 0x1d94ea thru 0x1de575 (0x508c bytes) 2024-09-21T15:33:11.827 INFO:teuthology.orchestra.run.smithi107.stdout:1354 writesame 0x97a8bc8 thru 0x97b3aa9 data_size 0x79(0xaee2 bytes) 2024-09-21T15:33:11.834 INFO:teuthology.orchestra.run.smithi107.stdout:1355 punch from 0x3dc2846 to 0x3dc95cc, (0x6d86 bytes) 2024-09-21T15:33:11.834 INFO:teuthology.orchestra.run.smithi107.stdout:1356 read 0x2790221 thru 0x279db61 (0xd941 bytes) 2024-09-21T15:33:11.834 INFO:teuthology.orchestra.run.smithi107.stdout:1357 read 0x30e4f90 thru 0x30e6d1a (0x1d8b bytes) 2024-09-21T15:33:11.841 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:11.839+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:11.842 INFO:teuthology.orchestra.run.smithi107.stdout:1359 trunc from 0xa3de90c to 0xdc9b991 2024-09-21T15:33:11.854 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.858 INFO:teuthology.orchestra.run.smithi107.stdout:1362 trunc from 0xdc9b991 to 0xc4c2117 2024-09-21T15:33:11.870 INFO:teuthology.orchestra.run.smithi107.stdout:1363 read 0x22e0d9 thru 0x234787 (0x66af bytes) 2024-09-21T15:33:11.871 INFO:teuthology.orchestra.run.smithi107.stdout:1364 writesame 0x552f97f thru 0x553205a data_size 0x6(0x26dc bytes) 2024-09-21T15:33:11.878 INFO:teuthology.orchestra.run.smithi107.stdout:1366 read 0x12d106b thru 0x12df049 (0xdfdf bytes) 2024-09-21T15:33:11.879 INFO:teuthology.orchestra.run.smithi107.stdout:1367 write 0x1aeec7c thru 0x1af3e17 (0x519c bytes) 2024-09-21T15:33:11.888 INFO:teuthology.orchestra.run.smithi107.stdout:1368 trunc from 0xc4c2117 to 0xec9ed61 2024-09-21T15:33:11.896 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:11.896 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:11.896 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:11.896 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:11.899 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.900 INFO:teuthology.orchestra.run.smithi107.stdout:1370 read 0x5d58346 thru 0x5d5f3ac (0x7067 bytes) 2024-09-21T15:33:11.900 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:11.900 INFO:teuthology.orchestra.run.smithi107.stdout:1372 writesame 0xa95a353 thru 0xa968775 data_size 0x1(0xe423 bytes) 2024-09-21T15:33:11.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:11.942 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:11.939+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:1374 clone 14 order 23 su 524288 sc 6 2024-09-21T15:33:12.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:12.042 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:12.039+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:12.081 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:12.081 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:12.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:12.143 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:12.143+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone13 from 0xec9ed61 (overlap 0x79c440) to 0x6d9706 2024-09-21T15:33:12.223 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.85477242, 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:12.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:12.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.826561244000001, 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:12.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.637261876, 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:12.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:12.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:12.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:12.239 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.505877184999999, 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:12.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:12.244 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.344779781, 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:12.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:12.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.344727234, 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:12.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.344679108999999, 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.257 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:12.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:12.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.338425093, 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:12.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.332282221, 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.771509239, 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.704484873, 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.569364741999999, 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528723951, 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528702599000001, 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528689382, 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528676575, 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528636168, 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.528611700000001, 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.509389712000001, 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.422086129, 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:12.331 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:12.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:12.332 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T15:33:12.333 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:12.331+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:12.333 INFO:teuthology.orchestra.run.smithi107.stdout:1375 trunc from 0xec9ed61 to 0x2c6bad6 2024-09-21T15:33:12.363 INFO:teuthology.orchestra.run.smithi107.stdout:1376 read 0x1be12ac thru 0x1beb30a (0xa05f bytes) 2024-09-21T15:33:12.364 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.364 INFO:teuthology.orchestra.run.smithi107.stdout:1381 read 0x1af757c thru 0x1b05b3f (0xe5c4 bytes) 2024-09-21T15:33:12.364 INFO:teuthology.orchestra.run.smithi107.stdout:1382 punch from 0x1a3caca to 0x1a42325, (0x585b bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1384 writesame 0xe200170 thru 0xe20339c data_size 0x7(0x322d bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1385 read 0x8a077aa thru 0x8a0ae0e (0x3665 bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1386 read 0x3b3e30e thru 0x3b4b98a (0xd67d bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1387 read 0xd072977 thru 0xd07e5a5 (0xbc2f bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1388 punch from 0x24eaab1 to 0x24efe87, (0x53d6 bytes) 2024-09-21T15:33:12.398 INFO:teuthology.orchestra.run.smithi107.stdout:1389 write 0x811d8e6 thru 0x81269f7 (0x9112 bytes) 2024-09-21T15:33:12.403 INFO:teuthology.orchestra.run.smithi107.stdout:1391 punch from 0x4c4ee35 to 0x4c5465d, (0x5828 bytes) 2024-09-21T15:33:12.409 INFO:teuthology.orchestra.run.smithi107.stdout:1393 trunc from 0xe20339d to 0xb984716 2024-09-21T15:33:12.420 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.420 INFO:teuthology.orchestra.run.smithi107.stdout:1395 read 0x3dd4a4f thru 0x3dd778b (0x2d3d bytes) 2024-09-21T15:33:12.420 INFO:teuthology.orchestra.run.smithi107.stdout:1396 read 0x7fa44da thru 0x7fb3b0b (0xf632 bytes) 2024-09-21T15:33:12.420 INFO:teuthology.orchestra.run.smithi107.stdout:1397 read 0xa9d3ae1 thru 0xa9dd640 (0x9b60 bytes) 2024-09-21T15:33:12.420 INFO:teuthology.orchestra.run.smithi107.stdout:1399 read 0x11fe82f thru 0x120236b (0x3b3d bytes) 2024-09-21T15:33:12.421 INFO:teuthology.orchestra.run.smithi107.stdout:1400 read 0x41b9a99 thru 0x41bef53 (0x54bb bytes) 2024-09-21T15:33:12.421 INFO:teuthology.orchestra.run.smithi107.stdout:1401 write 0x4630dfd thru 0x463b47e (0xa682 bytes) 2024-09-21T15:33:12.427 INFO:teuthology.orchestra.run.smithi107.stdout:1402 read 0xe642d7 thru 0xe72192 (0xdebc bytes) 2024-09-21T15:33:12.428 INFO:teuthology.orchestra.run.smithi107.stdout:1404 read 0x82bc595 thru 0x82c87d9 (0xc245 bytes) 2024-09-21T15:33:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:1405 writesame 0x31e0dea thru 0x31e656e data_size 0x1(0x5785 bytes) 2024-09-21T15:33:12.433 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:12.431+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:12.452 INFO:teuthology.orchestra.run.smithi107.stdout:1408 write 0x1ca229b thru 0x1ca4e58 (0x2bbe bytes) 2024-09-21T15:33:12.470 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:12.470 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:12.476 INFO:teuthology.orchestra.run.smithi107.stdout:1409 trunc from 0xb984716 to 0x2ea811f 2024-09-21T15:33:12.484 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:12.500 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.515 INFO:teuthology.orchestra.run.smithi107.stdout:1411 trunc from 0x2ea811f to 0x8aeb51c 2024-09-21T15:33:12.540 INFO:teuthology.orchestra.run.smithi107.stdout:1412 trunc from 0x8aeb51c to 0xe39ae9f 2024-09-21T15:33:12.551 INFO:teuthology.orchestra.run.smithi107.stdout:1413 read 0x698f8b thru 0x69d85e (0x48d4 bytes) 2024-09-21T15:33:12.555 INFO:teuthology.orchestra.run.smithi107.stdout:1414 trunc from 0xe39ae9f to 0xd204f5d 2024-09-21T15:33:12.565 INFO:teuthology.orchestra.run.smithi107.stdout:1415 read 0x818414f thru 0x8184de9 (0xc9b bytes) 2024-09-21T15:33:12.565 INFO:teuthology.orchestra.run.smithi107.stdout:1419 read 0x61ce378 thru 0x61d4351 (0x5fda bytes) 2024-09-21T15:33:12.565 INFO:teuthology.orchestra.run.smithi107.stdout:1420 punch from 0x24e34c0 to 0x24eba6e, (0x85ae bytes) 2024-09-21T15:33:12.571 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:12.571+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:12.573 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:12.576 INFO:teuthology.orchestra.run.smithi107.stdout:1422 write 0xd514272 thru 0xd51cd9f (0x8b2e bytes) 2024-09-21T15:33:12.583 INFO:teuthology.orchestra.run.smithi107.stdout:1424 write 0x3d55dd6 thru 0x3d5b8e1 (0x5b0c bytes) 2024-09-21T15:33:12.589 INFO:teuthology.orchestra.run.smithi107.stdout:1425 write 0x46af6f thru 0x47968c (0xe71e bytes) 2024-09-21T15:33:12.594 INFO:teuthology.orchestra.run.smithi107.stdout:1427 punch from 0x71525aa to 0x7159235, (0x6c8b bytes) 2024-09-21T15:33:12.594 INFO:teuthology.orchestra.run.smithi107.stdout:1429 read 0x1b2865f thru 0x1b2eaa9 (0x644b bytes) 2024-09-21T15:33:12.594 INFO:teuthology.orchestra.run.smithi107.stdout:1430 read 0x314cbe5 thru 0x315bb78 (0xef94 bytes) 2024-09-21T15:33:12.594 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.615 INFO:teuthology.orchestra.run.smithi107.stdout:1432 trunc from 0xd51cda0 to 0x342be8d 2024-09-21T15:33:12.635 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:12.631+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:12.646 INFO:teuthology.orchestra.run.smithi107.stdout:1433 write 0x41c4a8f thru 0x41c8332 (0x38a4 bytes) 2024-09-21T15:33:12.659 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:12.659 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:12.660 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:12.660 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:12.660 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:12.674 INFO:teuthology.orchestra.run.smithi107.stdout:1434 write 0xb67d8cb thru 0xb67f567 (0x1c9d bytes) 2024-09-21T15:33:12.675 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:12.678 INFO:teuthology.orchestra.run.smithi107.stdout:1436 writesame 0x65d4124 thru 0x65da518 data_size 0x1(0x63f5 bytes) 2024-09-21T15:33:12.705 INFO:teuthology.orchestra.run.smithi107.stdout:1437 write 0x7aa5043 thru 0x7aa56e7 (0x6a5 bytes) 2024-09-21T15:33:12.707 INFO:teuthology.orchestra.run.smithi107.stdout:1439 write 0xa2ad7c1 thru 0xa2afddc (0x261c bytes) 2024-09-21T15:33:12.712 INFO:teuthology.orchestra.run.smithi107.stdout:1440 read 0x774eb0d thru 0x7757a4b (0x8f3f bytes) 2024-09-21T15:33:12.724 INFO:teuthology.orchestra.run.smithi107.stdout:1443 trunc from 0xb67f568 to 0x36ee504 2024-09-21T15:33:12.732 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:12.733 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:12.736 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:12.735+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:12.739 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.750 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:12.750 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:12.751 INFO:teuthology.orchestra.run.smithi107.stdout:1445 write 0x6c441ee thru 0x6c46767 (0x257a bytes) 2024-09-21T15:33:12.753 INFO:teuthology.orchestra.run.smithi107.stdout:1446 write 0x301845c thru 0x302358f (0xb134 bytes) 2024-09-21T15:33:12.756 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.756 INFO:teuthology.orchestra.run.smithi107.stdout:1448 write 0x4fa18e0 thru 0x4fb1038 (0xf759 bytes) 2024-09-21T15:33:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:12.770 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:12.776 INFO:teuthology.orchestra.run.smithi107.stdout:1449 write 0xda6c20f thru 0xda6f1d1 (0x2fc3 bytes) 2024-09-21T15:33:12.780 INFO:teuthology.orchestra.run.smithi107.stdout:1451 write 0xb2041a4 thru 0xb20b0a8 (0x6f05 bytes) 2024-09-21T15:33:12.784 INFO:teuthology.orchestra.run.smithi107.stdout:1452 read 0x534c4bb thru 0x5352e13 (0x6959 bytes) 2024-09-21T15:33:12.784 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.784 INFO:teuthology.orchestra.run.smithi107.stdout:1458 punch from 0x9048e74 to 0x904af2d, (0x20b9 bytes) 2024-09-21T15:33:12.788 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:12.788 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:12.796 INFO:teuthology.orchestra.run.smithi107.stdout:1462 trunc from 0xda6f1d2 to 0x62a1af8 2024-09-21T15:33:12.808 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:12.809 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:12.810 INFO:teuthology.orchestra.run.smithi107.stdout:1463 write 0x45e19df thru 0x45e2310 (0x932 bytes) 2024-09-21T15:33:12.814 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.823 INFO:teuthology.orchestra.run.smithi107.stdout:1466 writesame 0x782b8eb thru 0x78356bf data_size 0x1(0x9dd5 bytes) 2024-09-21T15:33:12.825 INFO:teuthology.orchestra.run.smithi107.stdout:1467 punch from 0x63cb091 to 0x63cfaf9, (0x4a68 bytes) 2024-09-21T15:33:12.825 INFO:teuthology.orchestra.run.smithi107.stdout:1468 punch from 0x71ccef4 to 0x71d9dc9, (0xced5 bytes) 2024-09-21T15:33:12.825 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.825 INFO:teuthology.orchestra.run.smithi107.stdout:1471 read 0x1ee2d81 thru 0x1ee7d67 (0x4fe7 bytes) 2024-09-21T15:33:12.826 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.826 INFO:teuthology.orchestra.run.smithi107.stdout:1473 read 0x366c279 thru 0x366ddfb (0x1b83 bytes) 2024-09-21T15:33:12.826 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.826 INFO:teuthology.orchestra.run.smithi107.stdout:1477 write 0x310a23f thru 0x31140ad (0x9e6f bytes) 2024-09-21T15:33:12.828 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:12.828 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:12.828 INFO:teuthology.orchestra.run.smithi107.stdout:1478 punch from 0x5cb2843 to 0x5cbb326, (0x8ae3 bytes) 2024-09-21T15:33:12.834 INFO:teuthology.orchestra.run.smithi107.stdout:1479 trunc from 0x78356c0 to 0xb03767b 2024-09-21T15:33:12.837 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:12.835+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:12.842 INFO:teuthology.orchestra.run.smithi107.stdout:1480 punch from 0x4036164 to 0x403e2f6, (0x8192 bytes) 2024-09-21T15:33:12.842 INFO:teuthology.orchestra.run.smithi107.stdout:1481 writesame 0x617e9ec thru 0x6181acf data_size 0x2a(0x30e4 bytes) 2024-09-21T15:33:12.850 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:12.850 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:12.850 INFO:teuthology.orchestra.run.smithi107.stdout:1485 write 0x7e1fd3c thru 0x7e2d257 (0xd51c bytes) 2024-09-21T15:33:12.851 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:12.851 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:12.851 INFO:teuthology.orchestra.run.smithi107.stdout:1489 punch from 0x4b6fc52 to 0x4b7bde6, (0xc194 bytes) 2024-09-21T15:33:12.851 INFO:teuthology.orchestra.run.smithi107.stdout:1491 write 0xa28f8b3 thru 0xa29f2ee (0xfa3c bytes) 2024-09-21T15:33:12.856 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.856 INFO:teuthology.orchestra.run.smithi107.stdout:1496 trunc from 0xb03767b to 0xac72a1d 2024-09-21T15:33:12.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:12.867 INFO:teuthology.orchestra.run.smithi107.stdout:1498 write 0xaae1c5e thru 0xaae8b26 (0x6ec9 bytes) 2024-09-21T15:33:12.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:12.874 INFO:teuthology.orchestra.run.smithi107.stdout:1500 trunc from 0xac72a1d to 0x9f5b6cf 2024-09-21T15:33:12.883 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.883 INFO:teuthology.orchestra.run.smithi107.stdout:1503 writesame 0x1c33755 thru 0x1c35456 data_size 0x2f(0x1d02 bytes) 2024-09-21T15:33:12.885 INFO:teuthology.orchestra.run.smithi107.stdout:1504 read 0x226e89e thru 0x227627c (0x79df bytes) 2024-09-21T15:33:12.898 INFO:teuthology.orchestra.run.smithi107.stdout:1505 write 0xd52cb84 thru 0xd539a9b (0xcf18 bytes) 2024-09-21T15:33:12.901 INFO:teuthology.orchestra.run.smithi107.stdout:1506 write 0xc5d2a82 thru 0xc5d3723 (0xca2 bytes) 2024-09-21T15:33:12.907 INFO:teuthology.orchestra.run.smithi107.stdout:1509 read 0x10181bb thru 0x101c411 (0x4257 bytes) 2024-09-21T15:33:12.908 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.908 INFO:teuthology.orchestra.run.smithi107.stdout:1511 writesame 0x36b3132 thru 0x36b35e2 data_size 0x1(0x4b1 bytes) 2024-09-21T15:33:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:1512 read 0x5f4801 thru 0x5ff58a (0xad8a bytes) 2024-09-21T15:33:12.918 INFO:teuthology.orchestra.run.smithi107.stdout:1513 trunc from 0xd539a9c to 0x944cbc2 2024-09-21T15:33:12.933 INFO:teuthology.orchestra.run.smithi107.stdout:1514 punch from 0x1c06734 to 0x1c0ab0f, (0x43db bytes) 2024-09-21T15:33:12.933 INFO:teuthology.orchestra.run.smithi107.stdout:1515 punch from 0x40260be to 0x402bc7d, (0x5bbf bytes) 2024-09-21T15:33:12.933 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.933 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:12.933 INFO:teuthology.orchestra.run.smithi107.stdout:1519 write 0x7e6a6d9 thru 0x7e76716 (0xc03e bytes) 2024-09-21T15:33:12.934 INFO:teuthology.orchestra.run.smithi107.stdout:1520 read 0x872132b thru 0x872c172 (0xae48 bytes) 2024-09-21T15:33:12.938 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:12.935+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:12.941 INFO:teuthology.orchestra.run.smithi107.stdout:1521 trunc from 0x944cbc2 to 0x551b362 2024-09-21T15:33:12.976 INFO:teuthology.orchestra.run.smithi107.stdout:1522 write 0xaf4b169 thru 0xaf4e147 (0x2fdf bytes) 2024-09-21T15:33:12.991 INFO:teuthology.orchestra.run.smithi107.stdout:1523 trunc from 0xaf4e148 to 0x7438152 2024-09-21T15:33:12.996 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:33:12.999 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:12.999 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.015 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:13.021 INFO:teuthology.orchestra.run.smithi107.stdout:1524 trunc from 0x7438152 to 0xc4e1707 2024-09-21T15:33:13.030 INFO:teuthology.orchestra.run.smithi107.stdout:1525 write 0x5edb258 thru 0x5ee709a (0xbe43 bytes) 2024-09-21T15:33:13.032 INFO:teuthology.orchestra.run.smithi107.stdout:1528 write 0x53eaffc thru 0x53efc84 (0x4c89 bytes) 2024-09-21T15:33:13.034 INFO:teuthology.orchestra.run.smithi107.stdout:1529 read 0x6728ca8 thru 0x672a7a8 (0x1b01 bytes) 2024-09-21T15:33:13.034 INFO:teuthology.orchestra.run.smithi107.stdout:1530 read 0x18dd900 thru 0x18df740 (0x1e41 bytes) 2024-09-21T15:33:13.035 INFO:teuthology.orchestra.run.smithi107.stdout:1531 trunc from 0xc4e1707 to 0xc8a7afc 2024-09-21T15:33:13.038 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:13.035+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:13.042 INFO:teuthology.orchestra.run.smithi107.stdout:1532 write 0x48b2963 thru 0x48c0c2d (0xe2cb bytes) 2024-09-21T15:33:13.055 INFO:teuthology.orchestra.run.smithi107.stdout:1533 write 0xdf4cce5 thru 0xdf54af6 (0x7e12 bytes) 2024-09-21T15:33:13.062 INFO:teuthology.orchestra.run.smithi107.stdout:1534 trunc from 0xdf54af7 to 0xc9a4817 2024-09-21T15:33:13.076 INFO:teuthology.orchestra.run.smithi107.stdout:1535 trunc from 0xc9a4817 to 0xb62e0de 2024-09-21T15:33:13.091 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:13.091 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:13.091 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:13.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-21T15:33:13.106 INFO:teuthology.orchestra.run.smithi107.stdout:1536 writesame 0xe06b55d thru 0xe07a0e5 data_size 0x3(0xeb89 bytes) 2024-09-21T15:33:13.107 INFO:teuthology.orchestra.run.smithi107.stdout:1537 write 0x1472a19 thru 0x147e0a2 (0xb68a bytes) 2024-09-21T15:33:13.108 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.108 INFO:teuthology.orchestra.run.smithi107.stdout:1542 write 0x35bdc8e thru 0x35c6f27 (0x929a bytes) 2024-09-21T15:33:13.111 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.111 INFO:teuthology.orchestra.run.smithi107.stdout:1545 punch from 0x55556f5 to 0x5556ac0, (0x13cb bytes) 2024-09-21T15:33:13.111 INFO:teuthology.orchestra.run.smithi107.stdout:1546 punch from 0xcc57fc2 to 0xcc5ddd5, (0x5e13 bytes) 2024-09-21T15:33:13.111 INFO:teuthology.orchestra.run.smithi107.stdout:1547 write 0x316478d thru 0x316fc51 (0xb4c5 bytes) 2024-09-21T15:33:13.113 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.113 INFO:teuthology.orchestra.run.smithi107.stdout:1549 read 0x3a1dc69 thru 0x3a229ff (0x4d97 bytes) 2024-09-21T15:33:13.114 INFO:teuthology.orchestra.run.smithi107.stdout:1550 write 0x1c01656 thru 0x1c04d3b (0x36e6 bytes) 2024-09-21T15:33:13.121 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.135 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.135 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.137 INFO:teuthology.orchestra.run.smithi107.stdout:1552 trunc from 0xe07a0e6 to 0x3f31e29 2024-09-21T15:33:13.139 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:13.139+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:13.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:13.154 INFO:teuthology.orchestra.run.smithi107.stdout:1553 trunc from 0x3f31e29 to 0x1275c72 2024-09-21T15:33:13.170 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.170 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.171 INFO:teuthology.orchestra.run.smithi107.stdout:1557 read 0x57bd9e thru 0x5886d8 (0xc93b bytes) 2024-09-21T15:33:13.171 INFO:teuthology.orchestra.run.smithi107.stdout:1558 read 0x3af227 thru 0x3b53c1 (0x619b bytes) 2024-09-21T15:33:13.171 INFO:teuthology.orchestra.run.smithi107.stdout:1560 punch from 0x83e8e6 to 0x84a3d3, (0xbaed bytes) 2024-09-21T15:33:13.185 INFO:teuthology.orchestra.run.smithi107.stdout:1561 write 0x57d5359 thru 0x57e09a9 (0xb651 bytes) 2024-09-21T15:33:13.190 INFO:teuthology.orchestra.run.smithi107.stdout:1562 trunc from 0x57e09aa to 0x6b2d731 2024-09-21T15:33:13.198 INFO:teuthology.orchestra.run.smithi107.stdout:1566 read 0x33b5aaa thru 0x33b7596 (0x1aed bytes) 2024-09-21T15:33:13.198 INFO:teuthology.orchestra.run.smithi107.stdout:1567 writesame 0x1adeb87 thru 0x1aeab75 data_size 0x1f(0xbfef bytes) 2024-09-21T15:33:13.210 INFO:teuthology.orchestra.run.smithi107.stdout:1568 write 0x7e8ac25 thru 0x7e979d3 (0xcdaf bytes) 2024-09-21T15:33:13.222 INFO:teuthology.orchestra.run.smithi107.stdout:1569 write 0x834788b thru 0x83488d8 (0x104e bytes) 2024-09-21T15:33:13.225 INFO:teuthology.orchestra.run.smithi107.stdout:1571 read 0x20fd766 thru 0x21018bd (0x4158 bytes) 2024-09-21T15:33:13.239 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:13.235+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:13.240 INFO:teuthology.orchestra.run.smithi107.stdout:1575 write 0xd633b0e thru 0xd6350b8 (0x15ab bytes) 2024-09-21T15:33:13.244 INFO:teuthology.orchestra.run.smithi107.stdout:1577 read 0xfc1585 thru 0xfcb0d0 (0x9b4c bytes) 2024-09-21T15:33:13.249 INFO:teuthology.orchestra.run.smithi107.stdout:1578 trunc from 0xd6350b9 to 0xa50133a 2024-09-21T15:33:13.259 INFO:teuthology.orchestra.run.smithi107.stdout:1579 write 0x48200c4 thru 0x4829f61 (0x9e9e bytes) 2024-09-21T15:33:13.262 INFO:teuthology.orchestra.run.smithi107.stdout:1580 read 0xa46994e thru 0xa4703b7 (0x6a6a bytes) 2024-09-21T15:33:13.262 INFO:teuthology.orchestra.run.smithi107.stdout:1581 read 0x8dc6d93 thru 0x8dce638 (0x78a6 bytes) 2024-09-21T15:33:13.262 INFO:teuthology.orchestra.run.smithi107.stdout:1582 read 0x19b8992 thru 0x19c1a4c (0x90bb bytes) 2024-09-21T15:33:13.263 INFO:teuthology.orchestra.run.smithi107.stdout:1583 punch from 0x96e613 to 0x96e7b8, (0x1a5 bytes) 2024-09-21T15:33:13.263 INFO:teuthology.orchestra.run.smithi107.stdout:1584 write 0x438439f thru 0x43914be (0xd120 bytes) 2024-09-21T15:33:13.266 INFO:teuthology.orchestra.run.smithi107.stdout:1586 write 0x96925d2 thru 0x96989ba (0x63e9 bytes) 2024-09-21T15:33:13.269 INFO:teuthology.orchestra.run.smithi107.stdout:1587 write 0x7fdc5 thru 0x89695 (0x98d1 bytes) 2024-09-21T15:33:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:1589 write 0xa07c3df thru 0xa07f951 (0x3573 bytes) 2024-09-21T15:33:13.274 INFO:teuthology.orchestra.run.smithi107.stdout:1590 writesame 0xad9fd0 thru 0xadd95c data_size 0x3(0x398d bytes) 2024-09-21T15:33:13.277 INFO:teuthology.orchestra.run.smithi107.stdout:1591 trunc from 0xa50133a to 0xa3ab03b 2024-09-21T15:33:13.282 INFO:teuthology.orchestra.run.smithi107.stdout:1594 writesame 0x9244f01 thru 0x92452c7 data_size 0x1(0x3c7 bytes) 2024-09-21T15:33:13.289 INFO:teuthology.orchestra.run.smithi107.stdout:1595 writesame 0x19a821f thru 0x19b4f43 data_size 0x1(0xcd25 bytes) 2024-09-21T15:33:13.328 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.328 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.340 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:13.339+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:13.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:13.347 INFO:teuthology.orchestra.run.smithi107.stdout:1596 writesame 0x5389d7a thru 0x538f1d6 data_size 0x17(0x545d bytes) 2024-09-21T15:33:13.369 INFO:teuthology.orchestra.run.smithi107.stdout:1597 writesame 0xcfe57b9 thru 0xcfe631f data_size 0x3(0xb67 bytes) 2024-09-21T15:33:13.370 INFO:teuthology.orchestra.run.smithi107.stdout:1598 writesame 0x231a7ad thru 0x2326042 data_size 0x1(0xb896 bytes) 2024-09-21T15:33:13.423 INFO:teuthology.orchestra.run.smithi107.stdout:1600 write 0x51e4bc2 thru 0x51ef317 (0xa756 bytes) 2024-09-21T15:33:13.425 INFO:tasks.ceph.mon.a.smithi107.stderr:2024-09-21T15:33:13.423+0000 7fbc162d8640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-21T15:33:13.426 INFO:teuthology.orchestra.run.smithi107.stdout:1602 write 0x8fd4908 thru 0x8fd5436 (0xb2f bytes) 2024-09-21T15:33:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:1603 write 0x9809d89 thru 0x981958f (0xf807 bytes) 2024-09-21T15:33:13.433 INFO:teuthology.orchestra.run.smithi107.stdout:1604 write 0x8bcdfc8 thru 0x8bda036 (0xc06f bytes) 2024-09-21T15:33:13.438 INFO:teuthology.orchestra.run.smithi107.stdout:1605 punch from 0xb885515 to 0xb8863bb, (0xea6 bytes) 2024-09-21T15:33:13.438 INFO:teuthology.orchestra.run.smithi107.stdout:1607 read 0x21af04c thru 0x21bcaba (0xda6f bytes) 2024-09-21T15:33:13.438 INFO:teuthology.orchestra.run.smithi107.stdout:1608 write 0x5475493 thru 0x547d209 (0x7d77 bytes) 2024-09-21T15:33:13.440 INFO:teuthology.orchestra.run.smithi107.stdout:1609 writesame 0x237969c thru 0x237e68c data_size 0x1(0x4ff1 bytes) 2024-09-21T15:33:13.441 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:13.439+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:13.464 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.464 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.475 INFO:teuthology.orchestra.run.smithi107.stdout:1610 trunc from 0xcfe6320 to 0x286f849 2024-09-21T15:33:13.478 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:13.495 INFO:teuthology.orchestra.run.smithi107.stdout:1611 read 0x1f7a7fe thru 0x1f7f46e (0x4c71 bytes) 2024-09-21T15:33:13.526 INFO:teuthology.orchestra.run.smithi107.stdout:1612 write 0xafa2dae thru 0xafa5126 (0x2379 bytes) 2024-09-21T15:33:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:1614 read 0x717f4de thru 0x718463e (0x5161 bytes) 2024-09-21T15:33:13.532 INFO:teuthology.orchestra.run.smithi107.stdout:1615 writesame 0x1b6adc5 thru 0x1b6fcb2 data_size 0x1(0x4eee bytes) 2024-09-21T15:33:13.542 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:13.539+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:13.545 INFO:teuthology.orchestra.run.smithi107.stdout:1616 writesame 0x210cb4a thru 0x210de20 data_size 0xd(0x12d7 bytes) 2024-09-21T15:33:13.547 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.549 INFO:teuthology.orchestra.run.smithi107.stdout:1620 trunc from 0xafa5127 to 0xb6e48ed 2024-09-21T15:33:13.558 INFO:teuthology.orchestra.run.smithi107.stdout:1621 write 0x5349d8b thru 0x5352983 (0x8bf9 bytes) 2024-09-21T15:33:13.565 INFO:teuthology.orchestra.run.smithi107.stdout:1623 write 0x8ccacf4 thru 0x8cd94ca (0xe7d7 bytes) 2024-09-21T15:33:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:1624 read 0x801cc89 thru 0x801fb5c (0x2ed4 bytes) 2024-09-21T15:33:13.572 INFO:teuthology.orchestra.run.smithi107.stdout:1627 read 0xa943765 thru 0xa9446df (0xf7b bytes) 2024-09-21T15:33:13.593 INFO:teuthology.orchestra.run.smithi107.stdout:1628 trunc from 0xb6e48ed to 0x3b7a223 2024-09-21T15:33:13.643 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:13.639+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:13.647 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.647 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.650 INFO:teuthology.orchestra.run.smithi107.stdout:1630 writesame 0x9b38898 thru 0x9b3c48b data_size 0x6(0x3bf4 bytes) 2024-09-21T15:33:13.661 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:13.677 INFO:teuthology.orchestra.run.smithi107.stdout:1631 write 0xe7aa2aa thru 0xe7ab740 (0x1497 bytes) 2024-09-21T15:33:13.683 INFO:teuthology.orchestra.run.smithi107.stdout:1633 write 0xd2577a2 thru 0xd266f79 (0xf7d8 bytes) 2024-09-21T15:33:13.690 INFO:teuthology.orchestra.run.smithi107.stdout:1634 writesame 0xa775a33 thru 0xa77fbad data_size 0x1(0xa17b bytes) 2024-09-21T15:33:13.744 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:13.743+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:13.750 INFO:teuthology.orchestra.run.smithi107.stdout:1635 punch from 0xad443e8 to 0xad53251, (0xee69 bytes) 2024-09-21T15:33:13.751 INFO:teuthology.orchestra.run.smithi107.stdout:1637 writesame 0xcb543b2 thru 0xcb5f621 data_size 0x28(0xb270 bytes) 2024-09-21T15:33:13.758 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.758 INFO:teuthology.orchestra.run.smithi107.stdout:1640 punch from 0x7544d04 to 0x754eca8, (0x9fa4 bytes) 2024-09-21T15:33:13.759 INFO:teuthology.orchestra.run.smithi107.stdout:1641 trunc from 0xe7ab741 to 0xe27fdd8 2024-09-21T15:33:13.776 INFO:teuthology.orchestra.run.smithi107.stdout:1643 write 0x9b22a7b thru 0x9b29cf2 (0x7278 bytes) 2024-09-21T15:33:13.779 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.779 INFO:teuthology.orchestra.run.smithi107.stdout:1645 read 0xc9cdc5a thru 0xc9d837c (0xa723 bytes) 2024-09-21T15:33:13.780 INFO:teuthology.orchestra.run.smithi107.stdout:1646 read 0xda5f9fd thru 0xda67315 (0x7919 bytes) 2024-09-21T15:33:13.781 INFO:teuthology.orchestra.run.smithi107.stdout:1647 write 0xc957f28 thru 0xc961b66 (0x9c3f bytes) 2024-09-21T15:33:13.784 INFO:teuthology.orchestra.run.smithi107.stdout:1648 write 0x14ab180 thru 0x14b09c5 (0x5846 bytes) 2024-09-21T15:33:13.787 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.788 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:13.803 INFO:teuthology.orchestra.run.smithi107.stdout:1650 trunc from 0xe27fdd8 to 0x813420b 2024-09-21T15:33:13.845 INFO:teuthology.orchestra.run.smithi107.stdout:1652 write 0xb4eb11e thru 0xb4fa670 (0xf553 bytes) 2024-09-21T15:33:13.845 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:13.843+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:13.850 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.850 INFO:teuthology.orchestra.run.smithi107.stdout:1655 read 0x849f69b thru 0x84aa8a9 (0xb20f bytes) 2024-09-21T15:33:13.850 INFO:teuthology.orchestra.run.smithi107.stdout:1656 write 0xa06bb93 thru 0xa070a9a (0x4f08 bytes) 2024-09-21T15:33:13.855 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:13.868 INFO:teuthology.orchestra.run.smithi107.stdout:1658 trunc from 0xb4fa671 to 0x6bb8ee5 2024-09-21T15:33:13.887 INFO:teuthology.orchestra.run.smithi107.stdout:1659 read 0x3d9a6e9 thru 0x3da8955 (0xe26d bytes) 2024-09-21T15:33:13.887 INFO:teuthology.orchestra.run.smithi107.stdout:1661 trunc from 0x6bb8ee5 to 0x6bf9888 2024-09-21T15:33:13.897 INFO:teuthology.orchestra.run.smithi107.stdout:1662 write 0x5975b3e thru 0x59842a8 (0xe76b bytes) 2024-09-21T15:33:13.904 INFO:teuthology.orchestra.run.smithi107.stdout:1663 read 0x50d6301 thru 0x50ddf53 (0x7c53 bytes) 2024-09-21T15:33:13.925 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:13.925 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:13.938 INFO:teuthology.orchestra.run.smithi107.stdout:1664 write 0xdaa1915 thru 0xdaa1b00 (0x1ec bytes) 2024-09-21T15:33:13.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:13.944 INFO:teuthology.orchestra.run.smithi107.stdout:1665 punch from 0x75c9086 to 0x75d3e41, (0xadbb bytes) 2024-09-21T15:33:13.944 INFO:teuthology.orchestra.run.smithi107.stdout:1667 punch from 0xa3ddb3a to 0xa3e4a20, (0x6ee6 bytes) 2024-09-21T15:33:13.944 INFO:teuthology.orchestra.run.smithi107.stdout:1669 write 0x45ff10a thru 0x460c40d (0xd304 bytes) 2024-09-21T15:33:13.946 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:13.943+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:13.979 INFO:teuthology.orchestra.run.smithi107.stdout:1670 trunc from 0xdaa1b01 to 0x2b9aad0 2024-09-21T15:33:13.999 INFO:teuthology.orchestra.run.smithi107.stderr:marked out osd.2. 2024-09-21T15:33:14.025 INFO:teuthology.orchestra.run.smithi107.stdout:1671 trunc from 0x2b9aad0 to 0xaab1bef 2024-09-21T15:33:14.033 INFO:teuthology.orchestra.run.smithi107.stdout:1672 punch from 0x23f73a2 to 0x2402f30, (0xbb8e bytes) 2024-09-21T15:33:14.041 INFO:teuthology.orchestra.run.smithi107.stdout:1673 trunc from 0xaab1bef to 0x5e2f358 2024-09-21T15:33:14.046 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:14.043+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:14.063 INFO:teuthology.orchestra.run.smithi107.stdout:1675 write 0x93569a6 thru 0x935c296 (0x58f1 bytes) 2024-09-21T15:33:14.064 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.064 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.066 INFO:teuthology.orchestra.run.smithi107.stdout:1677 write 0x796c18d thru 0x796ca7a (0x8ee bytes) 2024-09-21T15:33:14.072 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:14.087 INFO:teuthology.orchestra.run.smithi107.stdout:1681 write 0xce86e06 thru 0xce8a652 (0x384d bytes) 2024-09-21T15:33:14.116 INFO:teuthology.orchestra.run.smithi107.stdout:1682 trunc from 0xce8a653 to 0x29b655f 2024-09-21T15:33:14.139 INFO:teuthology.orchestra.run.smithi107.stdout:1683 punch from 0x2227ed to 0x22bc73, (0x9486 bytes) 2024-09-21T15:33:14.139 INFO:teuthology.orchestra.run.smithi107.stdout:1684 punch from 0x196db6f to 0x196ec8b, (0x111c bytes) 2024-09-21T15:33:14.140 INFO:teuthology.orchestra.run.smithi107.stdout:1685 trunc from 0x29b655f to 0x346bc0d 2024-09-21T15:33:14.147 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:14.147+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:14.163 INFO:teuthology.orchestra.run.smithi107.stdout:1686 write 0x53aaab1 thru 0x53b5ee0 (0xb430 bytes) 2024-09-21T15:33:14.169 INFO:teuthology.orchestra.run.smithi107.stdout:1687 writesame 0x147d42f thru 0x147e364 data_size 0x21(0xf36 bytes) 2024-09-21T15:33:14.172 INFO:teuthology.orchestra.run.smithi107.stdout:1688 read 0x323de2e thru 0x323f0bd (0x1290 bytes) 2024-09-21T15:33:14.184 INFO:teuthology.orchestra.run.smithi107.stdout:1689 writesame 0x6285a06 thru 0x6286f90 data_size 0x1(0x158b bytes) 2024-09-21T15:33:14.197 INFO:teuthology.orchestra.run.smithi107.stdout:1690 write 0x77e6a5a thru 0x77f1ed6 (0xb47d bytes) 2024-09-21T15:33:14.202 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.202 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.204 INFO:teuthology.orchestra.run.smithi107.stdout:1691 write 0x14d4e80 thru 0x14de26b (0x93ec bytes) 2024-09-21T15:33:14.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:14.233 INFO:teuthology.orchestra.run.smithi107.stdout:1692 writesame 0xd88e23e thru 0xd893668 data_size 0x1(0x542b bytes) 2024-09-21T15:33:14.259 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:14.259+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:14.261 INFO:teuthology.orchestra.run.smithi107.stdout:1693 trunc from 0xd893669 to 0x1b78fc9 2024-09-21T15:33:14.280 INFO:teuthology.orchestra.run.smithi107.stdout:1694 punch from 0x1b33ce9 to 0x1b3db8c, (0x9ea3 bytes) 2024-09-21T15:33:14.308 INFO:teuthology.orchestra.run.smithi107.stdout:1695 writesame 0xaaf4c61 thru 0xaaf840c data_size 0xe(0x37ac bytes) 2024-09-21T15:33:14.308 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:1699 write 0xe49fca3 thru 0xe4aeac8 (0xee26 bytes) 2024-09-21T15:33:14.333 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.333 INFO:teuthology.orchestra.run.smithi107.stdout:1701 writesame 0xe3683a1 thru 0xe36a7c0 data_size 0x22(0x2420 bytes) 2024-09-21T15:33:14.338 INFO:teuthology.orchestra.run.smithi107.stdout:1702 read 0xe4002d5 thru 0xe40b9ea (0xb716 bytes) 2024-09-21T15:33:14.349 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:14.347+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:14.359 INFO:teuthology.orchestra.run.smithi107.stdout:1703 trunc from 0xe4aeac9 to 0x1b0bd9c 2024-09-21T15:33:14.381 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.381 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.388 INFO:teuthology.orchestra.run.smithi107.stdout:1704 read 0x11547ee thru 0x115f239 (0xaa4c bytes) 2024-09-21T15:33:14.388 INFO:teuthology.orchestra.run.smithi107.stdout:1706 punch from 0xe5554e to 0xe582c9, (0x2d7b bytes) 2024-09-21T15:33:14.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:14.408 INFO:teuthology.orchestra.run.smithi107.stdout:1707 write 0x4f68439 thru 0x4f6b9f8 (0x35c0 bytes) 2024-09-21T15:33:14.413 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.414 INFO:teuthology.orchestra.run.smithi107.stdout:1709 trunc from 0x4f6b9f9 to 0x3d32fcb 2024-09-21T15:33:14.427 INFO:teuthology.orchestra.run.smithi107.stdout:1712 punch from 0x33c95fc to 0x33d479d, (0xb1a1 bytes) 2024-09-21T15:33:14.431 INFO:teuthology.orchestra.run.smithi107.stdout:1713 trunc from 0x3d32fcb to 0x12bf7d8 2024-09-21T15:33:14.449 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:14.447+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:14.454 INFO:teuthology.orchestra.run.smithi107.stdout:1714 read 0xab2c50 thru 0xab4b11 (0x1ec2 bytes) 2024-09-21T15:33:14.487 INFO:teuthology.orchestra.run.smithi107.stdout:1715 write 0xd0d8a1b thru 0xd0da182 (0x1768 bytes) 2024-09-21T15:33:14.493 INFO:teuthology.orchestra.run.smithi107.stdout:1716 writesame 0x40a6b1e thru 0x40ad3b8 data_size 0x1(0x689b bytes) 2024-09-21T15:33:14.534 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.534 INFO:teuthology.orchestra.run.smithi107.stdout:1719 read 0x47e7d53 thru 0x47ed1ff (0x54ad bytes) 2024-09-21T15:33:14.534 INFO:teuthology.orchestra.run.smithi107.stdout:1722 write 0xc8d9f thru 0xd3515 (0xa777 bytes) 2024-09-21T15:33:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.537 INFO:teuthology.orchestra.run.smithi107.stdout:1723 write 0x346f18 thru 0x352cd4 (0xbdbd bytes) 2024-09-21T15:33:14.541 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.541 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.541 INFO:teuthology.orchestra.run.smithi107.stdout:1727 write 0x803882b thru 0x8039c08 (0x13de bytes) 2024-09-21T15:33:14.548 INFO:teuthology.orchestra.run.smithi107.stdout:1728 write 0xb999952 thru 0xb9a85cf (0xec7e bytes) 2024-09-21T15:33:14.550 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:14.547+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:14.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:14.569 INFO:teuthology.orchestra.run.smithi107.stdout:1729 write 0xd479569 thru 0xd48324c (0x9ce4 bytes) 2024-09-21T15:33:14.576 INFO:teuthology.orchestra.run.smithi107.stdout:1730 read 0x8cc4973 thru 0x8cd19cd (0xd05b bytes) 2024-09-21T15:33:14.597 INFO:teuthology.orchestra.run.smithi107.stdout:1731 trunc from 0xd48324d to 0x42e8d26 2024-09-21T15:33:14.628 INFO:teuthology.orchestra.run.smithi107.stdout:1733 trunc from 0x42e8d26 to 0xb6292c 2024-09-21T15:33:14.651 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:14.651+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:14.652 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.657 INFO:teuthology.orchestra.run.smithi107.stdout:1736 trunc from 0xb6292c to 0x2f17e4b 2024-09-21T15:33:14.668 INFO:teuthology.orchestra.run.smithi107.stdout:1739 read 0x1f00bde thru 0x1f0eb26 (0xdf49 bytes) 2024-09-21T15:33:14.669 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.692 INFO:teuthology.orchestra.run.smithi107.stdout:1742 trunc from 0x2f17e4b to 0xcd2f07b 2024-09-21T15:33:14.697 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.697 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.707 INFO:teuthology.orchestra.run.smithi107.stdout:1743 writesame 0x1e6e9d7 thru 0x1e7d2a3 data_size 0x1(0xe8cd bytes) 2024-09-21T15:33:14.711 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:14.752 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:14.751+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:14.781 INFO:teuthology.orchestra.run.smithi107.stdout:1744 punch from 0xb013820 to 0xb016653, (0x2e33 bytes) 2024-09-21T15:33:14.794 INFO:teuthology.orchestra.run.smithi107.stdout:1745 trunc from 0xcd2f07b to 0x5fb870f 2024-09-21T15:33:14.807 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.811 INFO:teuthology.orchestra.run.smithi107.stdout:1749 trunc from 0x5fb870f to 0x85b4e13 2024-09-21T15:33:14.838 INFO:teuthology.orchestra.run.smithi107.stdout:1751 writesame 0x994a481 thru 0x994e346 data_size 0x5(0x3ec6 bytes) 2024-09-21T15:33:14.838 INFO:teuthology.orchestra.run.smithi107.stdout:1752 write 0x854a297 thru 0x854ef47 (0x4cb1 bytes) 2024-09-21T15:33:14.843 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.843 INFO:teuthology.orchestra.run.smithi107.stdout:1754 read 0x1a76ee7 thru 0x1a79902 (0x2a1c bytes) 2024-09-21T15:33:14.853 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:14.851+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:14.855 INFO:teuthology.orchestra.run.smithi107.stdout:1755 trunc from 0x994e347 to 0x2ae805b 2024-09-21T15:33:14.879 INFO:teuthology.orchestra.run.smithi107.stdout:1757 punch from 0x2196289 to 0x219c94b, (0x66c2 bytes) 2024-09-21T15:33:14.882 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:14.883 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:14.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:14.905 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:14.914 INFO:teuthology.orchestra.run.smithi107.stdout:1758 write 0xdb52c11 thru 0xdb59813 (0x6c03 bytes) 2024-09-21T15:33:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:1761 read 0x262419e thru 0x262cbf7 (0x8a5a bytes) 2024-09-21T15:33:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:1762 read 0x5744f50 thru 0x5750b65 (0xbc16 bytes) 2024-09-21T15:33:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:1763 write 0x2e2a424 thru 0x2e326c1 (0x829e bytes) 2024-09-21T15:33:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:1764 punch from 0x68a0441 to 0x68aa272, (0x9e31 bytes) 2024-09-21T15:33:14.924 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.924 INFO:teuthology.orchestra.run.smithi107.stdout:1766 punch from 0x28a4946 to 0x28a7e6d, (0x3527 bytes) 2024-09-21T15:33:14.927 INFO:teuthology.orchestra.run.smithi107.stdout:1767 trunc from 0xdb59814 to 0xb0c5a78 2024-09-21T15:33:14.943 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.943 INFO:teuthology.orchestra.run.smithi107.stdout:1771 write 0x33db6cf thru 0x33e548f (0x9dc1 bytes) 2024-09-21T15:33:14.948 INFO:teuthology.orchestra.run.smithi107.stdout:1772 read 0x470f2d9 thru 0x471c05e (0xcd86 bytes) 2024-09-21T15:33:14.948 INFO:teuthology.orchestra.run.smithi107.stdout:1773 write 0x9a5a830 thru 0x9a65ab8 (0xb289 bytes) 2024-09-21T15:33:14.954 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:14.951+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:14.954 INFO:teuthology.orchestra.run.smithi107.stdout:1774 write 0x158823 thru 0x162a5f (0xa23d bytes) 2024-09-21T15:33:14.958 INFO:teuthology.orchestra.run.smithi107.stdout:1776 read 0x445196d thru 0x445d950 (0xbfe4 bytes) 2024-09-21T15:33:14.959 INFO:teuthology.orchestra.run.smithi107.stdout:1780 punch from 0x5389893 to 0x538c133, (0x28a0 bytes) 2024-09-21T15:33:14.959 INFO:teuthology.orchestra.run.smithi107.stdout:1781 read 0x6533c15 thru 0x653ff1e (0xc30a bytes) 2024-09-21T15:33:14.959 INFO:teuthology.orchestra.run.smithi107.stdout:1782 read 0x6f08db6 thru 0x6f11e26 (0x9071 bytes) 2024-09-21T15:33:14.959 INFO:teuthology.orchestra.run.smithi107.stdout:1783 read 0xef5290 thru 0xf04b40 (0xf8b1 bytes) 2024-09-21T15:33:14.959 INFO:teuthology.orchestra.run.smithi107.stdout:1784 writesame 0x1062538 thru 0x1064322 data_size 0x25(0x1deb bytes) 2024-09-21T15:33:14.963 INFO:teuthology.orchestra.run.smithi107.stdout:1785 punch from 0x4cb444 to 0x4d1cc9, (0x6885 bytes) 2024-09-21T15:33:14.963 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:14.977 INFO:teuthology.orchestra.run.smithi107.stdout:1787 write 0xce56f73 thru 0xce5ef07 (0x7f95 bytes) 2024-09-21T15:33:14.981 INFO:teuthology.orchestra.run.smithi107.stdout:1789 writesame 0x6f39b84 thru 0x6f41966 data_size 0x1(0x7de3 bytes) 2024-09-21T15:33:15.019 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:33:15.044 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:15.044 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:15.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:15.052 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:15.055 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:15.055+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:15.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:15.069 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:15.069 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:15.082 INFO:teuthology.orchestra.run.smithi107.stdout:1790 write 0xbf3200a thru 0xbf3a81b (0x8812 bytes) 2024-09-21T15:33:15.087 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:15.087 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:15.088 INFO:teuthology.orchestra.run.smithi107.stdout:1791 write 0x5b0f8df thru 0x5b16224 (0x6946 bytes) 2024-09-21T15:33:15.093 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:15.093 INFO:teuthology.orchestra.run.smithi107.stdout:1793 write 0x21513e0 thru 0x2158f3f (0x7b60 bytes) 2024-09-21T15:33:15.098 INFO:teuthology.orchestra.run.smithi107.stdout:1794 read 0x9b85aa4 thru 0x9b8f67d (0x9bda bytes) 2024-09-21T15:33:15.098 INFO:teuthology.orchestra.run.smithi107.stdout:1795 read 0x539a539 thru 0x539d1d0 (0x2c98 bytes) 2024-09-21T15:33:15.098 INFO:teuthology.orchestra.run.smithi107.stdout:1798 writesame 0x653bee5 thru 0x65447bd data_size 0x1(0x88d9 bytes) 2024-09-21T15:33:15.107 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:15.107 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:15.120 INFO:teuthology.orchestra.run.smithi107.stdout:1800 writesame 0x4424cc0 thru 0x442a0e0 data_size 0x3(0x5421 bytes) 2024-09-21T15:33:15.143 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:15.143 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:15.155 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:15.155+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:15.164 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:15.165 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:15.192 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:15.192 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:15.192 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:15.192 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.459614332000001, 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077593, 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731802+0000", 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731813+0000", 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1425e-05 2024-09-21T15:33:15.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731836+0000", 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2654999999999999e-05 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731865+0000", 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9289999999999999e-05 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.731972+0000", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106755 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.732008+0000", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6115999999999998e-05 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735799+0000", 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037913280000000001 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:48.735880+0000", 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0646e-05 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:666 3.0 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.649202710000001, 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055986830000000001, 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 666 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.198 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542214+0000", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542209+0000", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999948 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542230+0000", 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1667000000000001e-05 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542278+0000", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8158999999999997e-05 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542295+0000", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7e-05 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542430+0000", 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134342 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542674+0000", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000244625 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.542971+0000", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029688099999999998 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544915+0000", 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019438249999999999 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547571+0000", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002656416 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547713+0000", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014136899999999999 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547813+0000", 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9759000000000005e-05 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.202 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:717 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.613591561, 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040321619999999997, 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 717 2024-09-21T15:33:15.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577825+0000", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577824+0000", 2024-09-21T15:33:15.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577830+0000", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.46e-06 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577848+0000", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7802999999999999e-05 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577885+0000", 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6659000000000003e-05 2024-09-21T15:33:15.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581322+0000", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003437294 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581636+0000", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031343099999999997 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581857+0000", 2024-09-21T15:33:15.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000221645 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:720 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.613511125999999, 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046223590000000004, 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 720 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577906+0000", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577904+0000", 2024-09-21T15:33:15.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577912+0000", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4599999999999997e-06 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577933+0000", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1656999999999999e-05 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577942+0000", 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:15.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.581975+0000", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004032819 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582216+0000", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024039800000000001 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582528+0000", 2024-09-21T15:33:15.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031228799999999998 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:721 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.613465982000001, 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052225960000000004, 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 721 2024-09-21T15:33:15.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577951+0000", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577950+0000", 2024-09-21T15:33:15.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577954+0000", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.267e-06 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.577965+0000", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0316000000000001e-05 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578006+0000", 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1860000000000002e-05 2024-09-21T15:33:15.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582628+0000", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046216510000000001 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.582847+0000", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021899500000000001 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583173+0000", 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00032623400000000001 2024-09-21T15:33:15.214 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:724 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.613287754000002, 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055211540000000003, 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 724 2024-09-21T15:33:15.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578129+0000", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578127+0000", 2024-09-21T15:33:15.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578151+0000", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3366e-05 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578171+0000", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0543e-05 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578188+0000", 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6197000000000001e-05 2024-09-21T15:33:15.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583268+0000", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005080088 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583412+0000", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000144129 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583650+0000", 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000238244 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:725 3.0 3:05e0ca00:::rbd_data.11919c13aaf6.000000000000001a:5 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.613214104000001, 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060114970000000002, 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 725 2024-09-21T15:33:15.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578203+0000", 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578201+0000", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578208+0000", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3840000000000002e-06 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578229+0000", 2024-09-21T15:33:15.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1229999999999998e-05 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.578243+0000", 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3569e-05 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583747+0000", 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055046579999999999 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.583946+0000", 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019864500000000001 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.584214+0000", 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026819499999999999 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:878 3.1a e27/21 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head v 27'9, mlcod=26'7)", 2024-09-21T15:33:15.223 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.195207277000002, 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047062739999999999, 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996209+0000", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996208+0000", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996218+0000", 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0166000000000001e-05 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996236+0000", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7924e-05 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996246+0000", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5980000000000003e-06 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996296+0000", 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9886000000000003e-05 2024-09-21T15:33:15.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.996319+0000", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3045e-05 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000873+0000", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045538749999999998 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.000916+0000", 2024-09-21T15:33:15.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2969999999999997e-05 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:5584 3.1a e33/21 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head v 33'40, mlcod=32'38)", 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.139207198999999, 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053598999999999999, 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052209+0000", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.052208+0000", 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056205+0000", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003997202 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056221+0000", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6167e-05 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056231+0000", 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.55e-06 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056297+0000", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6231999999999999e-05 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.056329+0000", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1977999999999998e-05 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057476+0000", 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011471280000000001 2024-09-21T15:33:15.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:02.057569+0000", 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3178999999999997e-05 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7071 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2866592299999997, 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041211169999999997, 2024-09-21T15:33:15.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7071 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904757+0000", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904762+0000", 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904770+0000", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5879999999999998e-06 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.904777+0000", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6170000000000003e-06 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905465+0000", 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00068793299999999995 2024-09-21T15:33:15.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905513+0000", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7842e-05 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905832+0000", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031906099999999999 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908509+0000", 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026772739999999999 2024-09-21T15:33:15.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908784+0000", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027538999999999998 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908833+0000", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8958000000000001e-05 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908879+0000", 2024-09-21T15:33:15.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5105999999999998e-05 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7075 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2805909779999993, 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0045190079999999997, 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7075 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910826+0000", 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910825+0000", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910830+0000", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5090000000000004e-06 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910837+0000", 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2740000000000002e-06 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.910842+0000", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3789999999999997e-06 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911544+0000", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00070163500000000004 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.911629+0000", 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4837000000000001e-05 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912033+0000", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040393900000000002 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915180+0000", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031479189999999999 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915295+0000", 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114585 2024-09-21T15:33:15.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915324+0000", 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8923000000000001e-05 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915345+0000", 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0659e-05 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7321 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0958380230000007, 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053282879999999996, 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7321 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095579+0000", 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095578+0000", 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095581+0000", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2490000000000001e-06 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095588+0000", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4489999999999996e-06 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095591+0000", 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2909999999999999e-06 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096412+0000", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00082109700000000002 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096489+0000", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6391999999999996e-05 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096833+0000", 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034386699999999998 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100563+0000", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037307350000000002 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100810+0000", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024649300000000003 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100870+0000", 2024-09-21T15:33:15.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0096000000000002e-05 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100907+0000", 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7072e-05 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7322 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0958041880000007, 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0075079489999999999, 2024-09-21T15:33:15.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7322 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095612+0000", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095614+0000", 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095619+0000", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.904e-06 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095622+0000", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.847e-06 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097310+0000", 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016879639999999999 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097392+0000", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1959999999999995e-05 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097739+0000", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034728500000000002 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102987+0000", 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052481949999999998 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103053+0000", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5667000000000007e-05 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103082+0000", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8691999999999999e-05 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103120+0000", 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8568000000000001e-05 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.254 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7323 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.0957862820000006, 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0076219080000000002, 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7323 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095630+0000", 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095632+0000", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.074e-06 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095641+0000", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0920000000000004e-06 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.095649+0000", 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.174e-06 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098290+0000", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002641071 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098373+0000", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2400999999999999e-05 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098712+0000", 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00033931599999999999 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103134+0000", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044221629999999998 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103176+0000", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1721999999999998e-05 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103221+0000", 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5046000000000002e-05 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.103252+0000", 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1241999999999997e-05 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7330 3.0 3:04220828:::rbd_object_map.1191c706ae32:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.083341077, 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0057881219999999997, 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.260 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7330 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108076+0000", 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108075+0000", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108079+0000", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5979999999999999e-06 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108087+0000", 2024-09-21T15:33:15.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7270000000000007e-06 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108097+0000", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0287e-05 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108474+0000", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000376635 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108566+0000", 2024-09-21T15:33:15.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1529999999999999e-05 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108932+0000", 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036658600000000002 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111203+0000", 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0022704840000000001 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113816+0000", 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026135889999999999 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113846+0000", 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9471999999999998e-05 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113864+0000", 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7903000000000001e-05 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:7812 3.19 3:9b58d28c:::rbd_data.1191ebf845d7.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.2859535720000004, 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0085838960000000006, 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 7812 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905463+0000", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905462+0000", 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905467+0000", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2769999999999996e-06 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905478+0000", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0740999999999999e-05 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905483+0000", 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0300000000000001e-06 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905535+0000", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1724000000000001e-05 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.905597+0000", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1644999999999997e-05 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.907085+0000", 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001488917 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.913915+0000", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068291209999999996 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.913967+0000", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1981999999999998e-05 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.914002+0000", 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5743e-05 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.914047+0000", 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4613999999999997e-05 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10403 3.0 3:00861058:::rbd_data.1191f1341b63.000000000000002a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:11.903898+0000", 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.2875181950000001, 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.063810618999999999, 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10403 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903898+0000", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903898+0000", 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903895+0000", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903913+0000", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7316000000000002e-05 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903946+0000", 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3720000000000002e-05 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.903963+0000", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6269999999999998e-05 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.904090+0000", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000127389 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.904327+0000", 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00023715100000000001 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.918325+0000", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013998178999999999 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967541+0000", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049215729 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967562+0000", 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1166999999999999e-05 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967644+0000", 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2231999999999995e-05 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967709+0000", 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4611999999999997e-05 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.276 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:10403 3.0 e38/21 ondisk, result = 0)", 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:11.945745+0000", 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.24567161, 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021801317000000001, 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945745+0000", 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945745+0000", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945743+0000", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945754+0000", 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.08e-05 2024-09-21T15:33:15.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945787+0000", 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2688000000000003e-05 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.945805+0000", 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8377e-05 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967516+0000", 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021710855000000001 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967531+0000", 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4993e-05 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:11.967546+0000", 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5421e-05 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10851 3.19 3:9d65cf03:::rbd_object_map.11917b2c111e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.477636+0000", 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.7137808730000001, 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040211530000000004, 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10851 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.281 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477636+0000", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477636+0000", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477632+0000", 2024-09-21T15:33:15.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999957 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477659+0000", 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7316000000000001e-05 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477698+0000", 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9113999999999999e-05 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477717+0000", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8802999999999999e-05 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.477850+0000", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000132912 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.478121+0000", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027120099999999999 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:15.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.479173+0000", 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001051749 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.481040+0000", 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018670959999999999 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.481525+0000", 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00048497299999999998 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.481601+0000", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6196e-05 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.481657+0000", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5936000000000001e-05 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10905 3.19 3:9d65cf03:::rbd_object_map.11917b2c111e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.577573+0000", 2024-09-21T15:33:15.286 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 2.6138433399999998, 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0039572160000000004, 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10905 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577573+0000", 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577573+0000", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577572+0000", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577582+0000", 2024-09-21T15:33:15.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0701e-05 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577605+0000", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2433e-05 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577618+0000", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.295e-05 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.577747+0000", 2024-09-21T15:33:15.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012898799999999999 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.578011+0000", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026429099999999999 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.579054+0000", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010427990000000001 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.580432+0000", 2024-09-21T15:33:15.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0013785760000000001 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.581295+0000", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086231999999999997 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.581431+0000", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000136496 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.581531+0000", 2024-09-21T15:33:15.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9443000000000003e-05 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.292 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:15.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:15.294 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:15.295 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:15.291+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:15.393 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:15.391+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:15.454 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:15.454 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:15.454 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:15.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:15.512 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:15.511+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:15.596 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:15.595+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:15.633 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:15.633 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:15.633 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:15.633 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:15.633 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:15.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:15.732 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:15.731+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:15.798 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:15.795+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:15.815 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:641 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'8, mlcod=26'6)", 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.339000252000002, 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001951022, 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:15.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474657+0000", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474747+0000", 2024-09-21T15:33:15.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9559000000000001e-05 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474777+0000", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.991e-05 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474790+0000", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3246e-05 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474904+0000", 2024-09-21T15:33:15.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011387600000000001 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474973+0000", 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8847e-05 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476526+0000", 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015536180000000001 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476610+0000", 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3931999999999995e-05 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.269717916000001, 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:33:15.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3130 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'17, mlcod=28'14)", 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.989982798, 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018694930000000001, 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823676+0000", 2024-09-21T15:33:15.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823685+0000", 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823711+0000", 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6106e-05 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823726+0000", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4644000000000001e-05 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823829+0000", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102505 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823867+0000", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8606999999999998e-05 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825433+0000", 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015654810000000001 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825546+0000", 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113416 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.798801645000001, 2024-09-21T15:33:15.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:33:15.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.789011553000002, 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:33:15.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:33:15.836 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.775057513, 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.766890367999999, 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.841 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:33:15.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.766143969000002, 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:15.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:33:15.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:33:15.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.747752992999999, 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:33:15.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:33:15.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.686887053, 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.675216717000001, 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7071 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'176, mlcod=35'173)", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.9077487859999991, 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027688610000000001, 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905910+0000", 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905929+0000", 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9409999999999999e-05 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905943+0000", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4109e-05 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905950+0000", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905995+0000", 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4925000000000001e-05 2024-09-21T15:33:15.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.906020+0000", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4771000000000001e-05 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908648+0000", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002627622 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908680+0000", 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1795999999999998e-05 2024-09-21T15:33:15.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7075 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'178, mlcod=35'175)", 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.9015321870000008, 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030847729999999999, 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:15.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912134+0000", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5449999999999998e-06 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912146+0000", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1834999999999999e-05 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912153+0000", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7819999999999996e-06 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912186+0000", 2024-09-21T15:33:15.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3667999999999997e-05 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912199+0000", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2835e-05 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915184+0000", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029846120000000002 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915212+0000", 2024-09-21T15:33:15.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8242000000000002e-05 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7167416190000004, 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7158307629999996, 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:33:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7148648360000003, 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.879 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:33:15.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7329 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'206, mlcod=36'204)", 2024-09-21T15:33:15.883 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7055914150000007, 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003080213, 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108067+0000", 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108077+0000", 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7990000000000005e-06 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108098+0000", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1523999999999999e-05 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108109+0000", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0567999999999999e-05 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108153+0000", 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4187999999999998e-05 2024-09-21T15:33:15.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108175+0000", 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2089000000000001e-05 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111116+0000", 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029405490000000002 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111148+0000", 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2619000000000001e-05 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.7046608540000001, 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.888 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:33:15.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9864 3.9 e38/21 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head v 38'41, mlcod=37'39)", 2024-09-21T15:33:15.892 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.2981962879999998, 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026427170000000002, 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515462+0000", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516535+0000", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001073084 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516551+0000", 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5321000000000001e-05 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516560+0000", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8259999999999999e-06 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516610+0000", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0844999999999999e-05 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516625+0000", 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518072+0000", 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001446578 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518106+0000", 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4164e-05 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.896 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:10955 3.4 e39/21 3:21c0c826:::rbd_data.11917b2c111e.0000000000000016:head v 39'20, mlcod=38'18)", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.0859492959999999, 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020723389999999999, 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:15.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727708+0000", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727720+0000", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1742999999999999e-05 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727739+0000", 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8777e-05 2024-09-21T15:33:15.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727748+0000", 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727800+0000", 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2471e-05 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:15.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727824+0000", 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3913999999999998e-05 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.729746+0000", 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019218569999999999 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.729783+0000", 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6507000000000001e-05 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.900 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:15.901 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:15.901 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:15.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:15.902 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:15.899+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:33:16.003 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:15.999+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:16.077 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:16.077 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:16.077 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:16.078 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:16.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:16.136 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:16.135+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:16.204 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:16.203+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:16.212 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:16.212 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:16.212 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:16.212 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:16.213 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:16.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:16.258 INFO:teuthology.orchestra.run.smithi107.stdout:1801 write 0x32d8767 thru 0x32e7d54 (0xf5ee bytes) 2024-09-21T15:33:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:1804 write 0x52d06e8 thru 0x52d317b (0x2a94 bytes) 2024-09-21T15:33:16.274 INFO:teuthology.orchestra.run.smithi107.stdout:1805 write 0x6ca335c thru 0x6ca3adf (0x784 bytes) 2024-09-21T15:33:16.279 INFO:teuthology.orchestra.run.smithi107.stdout:1808 write 0x3bd090a thru 0x3bde72f (0xde26 bytes) 2024-09-21T15:33:16.300 INFO:teuthology.orchestra.run.smithi107.stdout:1810 trunc from 0xce5ef08 to 0x197de8f 2024-09-21T15:33:16.304 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:16.303+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:16.339 INFO:teuthology.orchestra.run.smithi107.stdout:1811 write 0x2913515 thru 0x291b3e0 (0x7ecc bytes) 2024-09-21T15:33:16.341 INFO:teuthology.orchestra.run.smithi107.stdout:1812 punch from 0x515c23 to 0x523eb7, (0xe294 bytes) 2024-09-21T15:33:16.341 INFO:teuthology.orchestra.run.smithi107.stdout:1813 read 0x7d790c thru 0x7e218c (0xa881 bytes) 2024-09-21T15:33:16.355 INFO:teuthology.orchestra.run.smithi107.stdout:1815 writesame 0x40c7fb5 thru 0x40d6ee2 data_size 0x9d(0xef2e bytes) 2024-09-21T15:33:16.357 INFO:teuthology.orchestra.run.smithi107.stdout:1816 trunc from 0x40d6ee3 to 0x54f95f3 2024-09-21T15:33:16.371 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.371 INFO:teuthology.orchestra.run.smithi107.stdout:1818 read 0x51c6cf1 thru 0x51d4e0f (0xe11f bytes) 2024-09-21T15:33:16.371 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.391 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.020794906999999, 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.992583731, 2024-09-21T15:33:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.803284363, 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.671899671999999, 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.510802267999999, 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.510749721, 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:16.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:16.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.510701596000001, 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:16.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:16.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.431 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.504447580000001, 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:16.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:16.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:16.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:16.438 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.498304707999999, 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:16.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:16.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:16.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.937531726, 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:16.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:16.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.870507360000001, 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:16.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.735387229000001, 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:16.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:16.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694746437999999, 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:16.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694725085999998, 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:16.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:16.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694711868999999, 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.475 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:16.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.479 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694699062000002, 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:16.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:16.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694658655000001, 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.485 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:16.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:16.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.694634186999998, 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:16.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.675412199, 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.495 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:16.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:16.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:16.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:16.501 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.588108616, 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:16.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:16.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:16.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:16.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.507 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:16.508 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:16.508 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:16.509 INFO:teuthology.orchestra.run.smithi107.stdout:1820 write 0xa109a84 thru 0xa1144a9 (0xaa26 bytes) 2024-09-21T15:33:16.509 INFO:teuthology.orchestra.run.smithi107.stdout:1821 write 0x3470b21 thru 0x34769a9 (0x5e89 bytes) 2024-09-21T15:33:16.509 INFO:teuthology.orchestra.run.smithi107.stdout:1822 punch from 0x23ae5bd to 0x23affe0, (0x1a23 bytes) 2024-09-21T15:33:16.509 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.509 INFO:teuthology.orchestra.run.smithi107.stdout:1824 read 0x4106266 thru 0x410acbc (0x4a57 bytes) 2024-09-21T15:33:16.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:16.512 INFO:teuthology.orchestra.run.smithi107.stdout:1825 write 0xe32665e thru 0xe333736 (0xd0d9 bytes) 2024-09-21T15:33:16.512 INFO:teuthology.orchestra.run.smithi107.stdout:1826 trunc from 0xe333737 to 0x89ff96a 2024-09-21T15:33:16.512 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1828 trunc from 0x89ff96a to 0x82bd482 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1830 punch from 0xe00d5b to 0xe0e1c7, (0xd46c bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1831 write 0xaef0218 thru 0xaefcf13 (0xccfc bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1832 punch from 0x2a2c277 to 0x2a2e617, (0x23a0 bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1833 read 0x30431f1 thru 0x3051dda (0xebea bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1834 read 0x8359264 thru 0x835c88e (0x362b bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1835 read 0x76747e7 thru 0x7678e72 (0x468c bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1837 write 0x7ad4c0b thru 0x7adda7c (0x8e72 bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1839 read 0x18d0060 thru 0x18d9b2c (0x9acd bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1841 write 0x4e76896 thru 0x4e7cf9c (0x6707 bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1844 punch from 0x42c5be3 to 0x42c7c39, (0x2056 bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1845 read 0x949bdbe thru 0x94a65fb (0xa83e bytes) 2024-09-21T15:33:16.513 INFO:teuthology.orchestra.run.smithi107.stdout:1848 read 0x8686a8c thru 0x868a440 (0x39b5 bytes) 2024-09-21T15:33:16.514 INFO:teuthology.orchestra.run.smithi107.stdout:1849 writesame 0x6d3e7b thru 0x6ddd80 data_size 0x45(0x9f06 bytes) 2024-09-21T15:33:16.514 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:16.507+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:16.520 INFO:teuthology.orchestra.run.smithi107.stdout:1851 trunc from 0xaefcf14 to 0x970de5e 2024-09-21T15:33:16.536 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.536 INFO:teuthology.orchestra.run.smithi107.stdout:1854 write 0x67b6d4d thru 0x67b915a (0x240e bytes) 2024-09-21T15:33:16.556 INFO:teuthology.orchestra.run.smithi107.stdout:1856 write 0x9bddc37 thru 0x9be9a68 (0xbe32 bytes) 2024-09-21T15:33:16.564 INFO:teuthology.orchestra.run.smithi107.stdout:1857 read 0x5d820d0 thru 0x5d8ddef (0xbd20 bytes) 2024-09-21T15:33:16.580 INFO:teuthology.orchestra.run.smithi107.stdout:1858 write 0xa6c0945 thru 0xa6cb7b5 (0xae71 bytes) 2024-09-21T15:33:16.586 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.586 INFO:teuthology.orchestra.run.smithi107.stdout:1861 punch from 0x6d74136 to 0x6d77cf2, (0x3bbc bytes) 2024-09-21T15:33:16.586 INFO:teuthology.orchestra.run.smithi107.stdout:1862 write 0x10656e thru 0x109c98 (0x372b bytes) 2024-09-21T15:33:16.605 INFO:teuthology.orchestra.run.smithi107.stdout:1863 write 0xca23da5 thru 0xca29819 (0x5a75 bytes) 2024-09-21T15:33:16.610 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:16.607+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:16.618 INFO:teuthology.orchestra.run.smithi107.stdout:1864 trunc from 0xca2981a to 0xebf34a8 2024-09-21T15:33:16.629 INFO:teuthology.orchestra.run.smithi107.stdout:1866 writesame 0xd394712 thru 0xd399647 data_size 0x1(0x4f36 bytes) 2024-09-21T15:33:16.647 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:16.647 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:16.647 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:16.647 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:16.651 INFO:teuthology.orchestra.run.smithi107.stdout:1867 write 0x24774e3 thru 0x247c1c0 (0x4cde bytes) 2024-09-21T15:33:16.654 INFO:teuthology.orchestra.run.smithi107.stdout:1868 read 0x28b2422 thru 0x28b6a8e (0x466d bytes) 2024-09-21T15:33:16.655 INFO:teuthology.orchestra.run.smithi107.stdout:1871 write 0x1d474ff thru 0x1d47b4c (0x64e bytes) 2024-09-21T15:33:16.662 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:16.678 INFO:teuthology.orchestra.run.smithi107.stdout:1872 trunc from 0xebf34a8 to 0x7246b5d 2024-09-21T15:33:16.710 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:16.707+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:16.717 INFO:teuthology.orchestra.run.smithi107.stdout:1873 write 0x91db152 thru 0x91e9e8c (0xed3b bytes) 2024-09-21T15:33:16.727 INFO:teuthology.orchestra.run.smithi107.stdout:1874 read 0x459391 thru 0x45fb6e (0x67de bytes) 2024-09-21T15:33:16.727 INFO:teuthology.orchestra.run.smithi107.stdout:1875 punch from 0x3f4d0b8 to 0x3f52417, (0x535f bytes) 2024-09-21T15:33:16.728 INFO:teuthology.orchestra.run.smithi107.stdout:1877 punch from 0x4202a84 to 0x42071b4, (0x4730 bytes) 2024-09-21T15:33:16.759 INFO:teuthology.orchestra.run.smithi107.stdout:1879 writesame 0xdef9125 thru 0xdf070f1 data_size 0x2f(0xdfcd bytes) 2024-09-21T15:33:16.759 INFO:teuthology.orchestra.run.smithi107.stdout:1880 read 0xdb83ffa thru 0xdb8ef7e (0xaf85 bytes) 2024-09-21T15:33:16.759 INFO:teuthology.orchestra.run.smithi107.stdout:1881 read 0x94783c1 thru 0x94877a8 (0xf3e8 bytes) 2024-09-21T15:33:16.759 INFO:teuthology.orchestra.run.smithi107.stdout:1882 writesame 0x46e04d5 thru 0x46e72bc data_size 0x4(0x6de8 bytes) 2024-09-21T15:33:16.768 INFO:teuthology.orchestra.run.smithi107.stdout:1884 write 0x9189bf1 thru 0x918f837 (0x5c47 bytes) 2024-09-21T15:33:16.770 INFO:teuthology.orchestra.run.smithi107.stdout:1887 write 0x54fa2ca thru 0x5503715 (0x944c bytes) 2024-09-21T15:33:16.777 INFO:teuthology.orchestra.run.smithi107.stdout:1888 read 0x3459a35 thru 0x345f6fd (0x5cc9 bytes) 2024-09-21T15:33:16.778 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.778 INFO:teuthology.orchestra.run.smithi107.stdout:1890 writesame 0x4af67d2 thru 0x4afb716 data_size 0xd(0x4f45 bytes) 2024-09-21T15:33:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:1891 read 0x9fb71c0 thru 0x9fc60c6 (0xef07 bytes) 2024-09-21T15:33:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:1892 read 0xa3cf0f9 thru 0xa3d4d4a (0x5c52 bytes) 2024-09-21T15:33:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:1893 read 0xb410ae0 thru 0xb41d61d (0xcb3e bytes) 2024-09-21T15:33:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:1894 read 0x262ca6d thru 0x2638cb9 (0xc24d bytes) 2024-09-21T15:33:16.788 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.788 INFO:teuthology.orchestra.run.smithi107.stdout:1897 read 0x8afb7a1 thru 0x8b08272 (0xcad2 bytes) 2024-09-21T15:33:16.788 INFO:teuthology.orchestra.run.smithi107.stdout:1898 punch from 0x7ccde1e to 0x7ccef24, (0x1106 bytes) 2024-09-21T15:33:16.788 INFO:teuthology.orchestra.run.smithi107.stdout:1899 write 0x8f0fd98 thru 0x8f1ccf8 (0xcf61 bytes) 2024-09-21T15:33:16.796 INFO:teuthology.orchestra.run.smithi107.stdout:1900 punch from 0x5471e02 to 0x5473f07, (0x2105 bytes) 2024-09-21T15:33:16.796 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:16.796 INFO:teuthology.orchestra.run.smithi107.stdout:1903 write 0x52c43ab thru 0x52cadcf (0x6a25 bytes) 2024-09-21T15:33:16.799 INFO:teuthology.orchestra.run.smithi107.stdout:1904 read 0xd2324cf thru 0xd241cac (0xf7de bytes) 2024-09-21T15:33:16.799 INFO:teuthology.orchestra.run.smithi107.stdout:1905 punch from 0xa93ddfa to 0xa93f826, (0x1a2c bytes) 2024-09-21T15:33:16.799 INFO:teuthology.orchestra.run.smithi107.stdout:1906 read 0x2209412 thru 0x2214271 (0xae60 bytes) 2024-09-21T15:33:16.800 INFO:teuthology.orchestra.run.smithi107.stdout:1907 write 0x5d8b6e3 thru 0x5d91dfd (0x671b bytes) 2024-09-21T15:33:16.803 INFO:teuthology.orchestra.run.smithi107.stdout:1908 write 0x37fb008 thru 0x380026e (0x5267 bytes) 2024-09-21T15:33:16.811 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:16.807+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:16.831 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:16.831 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:16.831 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:16.831 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:16.831 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:16.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:16.911 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:16.907+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:17.012 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:17.011+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:17.014 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.280415232999999, 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00570851, 2024-09-21T15:33:17.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732737+0000", 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732760+0000", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2650000000000002e-05 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732808+0000", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8455999999999998e-05 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732829+0000", 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0837000000000001e-05 2024-09-21T15:33:17.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732965+0000", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013561899999999999 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733187+0000", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022187800000000001 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733408+0000", 2024-09-21T15:33:17.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022163799999999999 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.734741+0000", 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013323199999999999 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738291+0000", 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003550802 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738376+0000", 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4519999999999997e-05 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738449+0000", 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2805999999999994e-05 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.337579498, 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.022 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:17.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:17.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:17.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:878 3.1a 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.016151391000001, 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072331519999999996, 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 878 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997002+0000", 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997018+0000", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6682999999999999e-05 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997047+0000", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8503000000000001e-05 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997067+0000", 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9962999999999998e-05 2024-09-21T15:33:17.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997172+0000", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000104676 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997378+0000", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000206465 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997965+0000", 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000586569 2024-09-21T15:33:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.999686+0000", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001720877 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004149+0000", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044633499999999996 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004192+0000", 2024-09-21T15:33:17.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3501000000000002e-05 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004237+0000", 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4848000000000002e-05 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.221482145, 2024-09-21T15:33:17.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:17.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.038 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.184902031, 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:17.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.165973337, 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1319 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:17.049 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.062366763, 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060901970000000003, 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950786+0000", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950804+0000", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8017e-05 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950836+0000", 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1328999999999998e-05 2024-09-21T15:33:17.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950855+0000", 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9714000000000001e-05 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950976+0000", 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120635 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951033+0000", 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7469999999999997e-05 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956831+0000", 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057975910000000004 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956879+0000", 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7938000000000001e-05 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3381 3.9 3:91379144:::rbd_object_map.1191cae494be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.964324170000001, 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066017230000000003, 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3381 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048830+0000", 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:17.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048837+0000", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048850+0000", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2991e-05 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048858+0000", 2024-09-21T15:33:17.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049065+0000", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020716899999999999 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049141+0000", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6222000000000003e-05 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049454+0000", 2024-09-21T15:33:17.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031307500000000001 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.050206+0000", 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00075177000000000002 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055362+0000", 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00515586 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055404+0000", 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1974999999999997e-05 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055433+0000", 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9254000000000001e-05 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.426224306000002, 2024-09-21T15:33:17.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:17.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3986 3.9 3:931f7677:::rbd_data.119117b26f37.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.377846413, 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077269110000000004, 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3986 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635306+0000", 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635321+0000", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.431e-05 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635351+0000", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0633000000000003e-05 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635373+0000", 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.139e-05 2024-09-21T15:33:17.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635463+0000", 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599000000000005e-05 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635688+0000", 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022439500000000001 2024-09-21T15:33:17.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.637251+0000", 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001563348 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642822+0000", 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055710539999999998 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642898+0000", 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5783000000000003e-05 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642982+0000", 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3814999999999996e-05 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.643036+0000", 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4187000000000001e-05 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:17.070 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.291001912999999, 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.287973182999998, 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:17.076 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:17.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:17.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:17.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4247 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'77, mlcod=31'75)", 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.177599667999999, 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069077940000000001, 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835554+0000", 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835571+0000", 2024-09-21T15:33:17.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:33:17.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835605+0000", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.375e-05 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835624+0000", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9606000000000002e-05 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835747+0000", 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122846 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835802+0000", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4703000000000001e-05 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842381+0000", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065796170000000003 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842464+0000", 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2169000000000006e-05 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.086 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4249 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'22, mlcod=31'20)", 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.177298267000001, 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006656276, 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:17.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835856+0000", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835873+0000", 2024-09-21T15:33:17.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7397e-05 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835898+0000", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4995999999999999e-05 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835914+0000", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.562e-05 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836036+0000", 2024-09-21T15:33:17.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121886 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836082+0000", 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6430000000000001e-05 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842450+0000", 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063680170000000001 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842513+0000", 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3377e-05 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.98976785, 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:17.091 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:17.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.939182213, 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.097 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:17.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9864 3.9 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.4977495640000003, 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067473979999999999, 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9864 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:17.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515403+0000", 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516330+0000", 2024-09-21T15:33:17.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00092668600000000002 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516362+0000", 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1402999999999997e-05 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516377+0000", 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.523e-05 2024-09-21T15:33:17.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516476+0000", 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9205999999999999e-05 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516715+0000", 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023897400000000001 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.517256+0000", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054115999999999995 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.520752+0000", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003495976 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.521884+0000", 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001132384 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522029+0000", 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014489599999999999 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522153+0000", 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000123936 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.108 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9922 3.1a 3:5f081d68:::rbd_data.1191f1341b63.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.383295221, 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060256199999999998, 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9922 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629858+0000", 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629874+0000", 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5557e-05 2024-09-21T15:33:17.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629902+0000", 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8065999999999999e-05 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629916+0000", 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:17.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630016+0000", 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9822999999999996e-05 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630166+0000", 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015055900000000001 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.631290+0000", 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001123474 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635698+0000", 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044081859999999997 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635721+0000", 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096999999999999e-05 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635797+0000", 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6229000000000005e-05 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635886+0000", 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8578999999999993e-05 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:17.116 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.5774717479999998, 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:17.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11989 3.9 3:9126918e:::rbd_data.11917b2c111e.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.70691126100000001, 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00572151, 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11989 2024-09-21T15:33:17.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:17.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306242+0000", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306255+0000", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.308e-05 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306288+0000", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2736e-05 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:17.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306309+0000", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0939999999999999e-05 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307487+0000", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011785350000000001 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307693+0000", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020588799999999999 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:17.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.308075+0000", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038175999999999998 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.310669+0000", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002594 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311745+0000", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001075398 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:17.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311865+0000", 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120353 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311966+0000", 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100846 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:17.127 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:17.129 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:17.129 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:17.127+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:17.229 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:17.227+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:17.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:17.300 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:17.300 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:17.300 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:17.300 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:17.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:17.340 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:17.339+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:17.431 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:17.427+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:17.450 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:17.467 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:17.467 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:17.482 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:17.482 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:17.483 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:17.483 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:17.483 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:17.483 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:17.483 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:17.497 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:17.498 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:17.498 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:17.518 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:17.518 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:17.531 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:17.527+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:17.535 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:17.535 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:17.552 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:17.552 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:17.569 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:17.569 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:17.621 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:17.621 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:17.632 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:17.631+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:17.636 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:17.733 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:17.731+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:17.736 INFO:teuthology.orchestra.run.smithi107.stdout:1911 trunc from 0xdf070f2 to 0xd0a49b5 2024-09-21T15:33:17.755 INFO:teuthology.orchestra.run.smithi107.stdout:1912 read 0x32cdae9 thru 0x32d01a0 (0x26b8 bytes) 2024-09-21T15:33:17.760 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:17.760 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:17.766 INFO:teuthology.orchestra.run.smithi107.stdout:1913 trunc from 0xd0a49b5 to 0x937763c 2024-09-21T15:33:17.775 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:17.780 INFO:teuthology.orchestra.run.smithi107.stdout:1914 read 0x57fb61 thru 0x583302 (0x37a2 bytes) 2024-09-21T15:33:17.782 INFO:teuthology.orchestra.run.smithi107.stdout:1915 trunc from 0x937763c to 0x9a5f116 2024-09-21T15:33:17.792 INFO:teuthology.orchestra.run.smithi107.stdout:1916 read 0xc83b45 thru 0xc85c33 (0x20ef bytes) 2024-09-21T15:33:17.793 INFO:teuthology.orchestra.run.smithi107.stdout:1917 writesame 0x237412f thru 0x2377f42 data_size 0x1d(0x3e14 bytes) 2024-09-21T15:33:17.797 INFO:teuthology.orchestra.run.smithi107.stdout:1918 write 0x514f444 thru 0x515ba97 (0xc654 bytes) 2024-09-21T15:33:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:1920 read 0x20e12b9 thru 0x20ea8e5 (0x962d bytes) 2024-09-21T15:33:17.819 INFO:teuthology.orchestra.run.smithi107.stdout:1921 writesame 0xc2be576 thru 0xc2cd9a8 data_size 0x1(0xf433 bytes) 2024-09-21T15:33:17.835 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:17.831+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:17.836 INFO:teuthology.orchestra.run.smithi107.stdout:1923 trunc from 0xc2cd9a9 to 0x4ee0d30 2024-09-21T15:33:17.860 INFO:teuthology.orchestra.run.smithi107.stdout:1924 read 0x271dc4c thru 0x2721b2d (0x3ee2 bytes) 2024-09-21T15:33:17.861 INFO:teuthology.orchestra.run.smithi107.stdout:1925 read 0x63fd8e thru 0x64d249 (0xd4bc bytes) 2024-09-21T15:33:17.862 INFO:teuthology.orchestra.run.smithi107.stdout:1926 write 0x1080518 thru 0x108800c (0x7af5 bytes) 2024-09-21T15:33:17.866 INFO:teuthology.orchestra.run.smithi107.stdout:1927 write 0x4c106d7 thru 0x4c1f905 (0xf22f bytes) 2024-09-21T15:33:17.869 INFO:teuthology.orchestra.run.smithi107.stdout:1928 read 0x219b20d thru 0x21a2ea8 (0x7c9c bytes) 2024-09-21T15:33:17.886 INFO:teuthology.orchestra.run.smithi107.stdout:1929 trunc from 0x4ee0d30 to 0xc1c0ce7 2024-09-21T15:33:17.898 INFO:teuthology.orchestra.run.smithi107.stdout:1931 read 0x96f553f thru 0x970433c (0xedfe bytes) 2024-09-21T15:33:17.898 INFO:teuthology.orchestra.run.smithi107.stdout:1932 punch from 0x4d04645 to 0x4d1271d, (0xe0d8 bytes) 2024-09-21T15:33:17.898 INFO:teuthology.orchestra.run.smithi107.stdout:1933 writesame 0x1ebe981 thru 0x1ec6449 data_size 0x11(0x7ac9 bytes) 2024-09-21T15:33:17.903 INFO:teuthology.orchestra.run.smithi107.stdout:1935 write 0x80fa84a thru 0x8106648 (0xbdff bytes) 2024-09-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi107.stdout:1936 read 0x78be67a thru 0x78c2338 (0x3cbf bytes) 2024-09-21T15:33:17.912 INFO:teuthology.orchestra.run.smithi107.stdout:1938 write 0x262cd8b thru 0x26384f6 (0xb76c bytes) 2024-09-21T15:33:17.918 INFO:teuthology.orchestra.run.smithi107.stdout:1939 trunc from 0xc1c0ce7 to 0xd52f9f8 2024-09-21T15:33:17.929 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:17.930 INFO:teuthology.orchestra.run.smithi107.stdout:1942 writesame 0xa29e211 thru 0xa2adb10 data_size 0xc0(0xf900 bytes) 2024-09-21T15:33:17.935 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:17.935+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:17.946 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:17.946 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:1943 trunc from 0xd52f9f8 to 0x919e715 2024-09-21T15:33:17.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:1945 read 0x49950f9 thru 0x4997b13 (0x2a1b bytes) 2024-09-21T15:33:17.985 INFO:teuthology.orchestra.run.smithi107.stdout:1946 write 0xcf91655 thru 0xcf98d6d (0x7719 bytes) 2024-09-21T15:33:18.013 INFO:teuthology.orchestra.run.smithi107.stdout:1947 trunc from 0xcf98d6e to 0x5005120 2024-09-21T15:33:18.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:33:18.038 INFO:teuthology.orchestra.run.smithi107.stdout:1948 read 0xfaee98 thru 0xfb09fb (0x1b64 bytes) 2024-09-21T15:33:18.039 INFO:teuthology.orchestra.run.smithi107.stdout:1952 punch from 0x2d4754b to 0x2d4e6fe, (0x71b3 bytes) 2024-09-21T15:33:18.051 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:18.051+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:18.052 INFO:teuthology.orchestra.run.smithi107.stdout:1953 write 0x6e18bef thru 0x6e239ca (0xaddc bytes) 2024-09-21T15:33:18.056 INFO:teuthology.orchestra.run.smithi107.stdout:1954 punch from 0x5f560e6 to 0x5f565bc, (0x4d6 bytes) 2024-09-21T15:33:18.068 INFO:teuthology.orchestra.run.smithi107.stdout:1955 trunc from 0x6e239cb to 0xa4639d 2024-09-21T15:33:18.086 INFO:teuthology.orchestra.run.smithi107.stdout:1956 read 0x6fa348 thru 0x7029bc (0x8675 bytes) 2024-09-21T15:33:18.087 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.087 INFO:teuthology.orchestra.run.smithi107.stdout:1961 read 0x67dd2a thru 0x68b3d5 (0xd6ac bytes) 2024-09-21T15:33:18.088 INFO:teuthology.orchestra.run.smithi107.stdout:1962 read 0x31c32b thru 0x320ba2 (0x4878 bytes) 2024-09-21T15:33:18.088 INFO:teuthology.orchestra.run.smithi107.stdout:1963 read 0x627c6f thru 0x62cb23 (0x4eb5 bytes) 2024-09-21T15:33:18.108 INFO:teuthology.orchestra.run.smithi107.stdout:1964 writesame 0x67c0138 thru 0x67ceaff data_size 0x4(0xe9c8 bytes) 2024-09-21T15:33:18.120 INFO:teuthology.orchestra.run.smithi107.stdout:1965 writesame 0x68c58d1 thru 0x68cf9a2 data_size 0x5(0xa0d2 bytes) 2024-09-21T15:33:18.121 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.121 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.135 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.136 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.137 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:18.135+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:18.142 INFO:teuthology.orchestra.run.smithi107.stdout:1970 writesame 0xd27e779 thru 0xd2890ba data_size 0x23(0xa942 bytes) 2024-09-21T15:33:18.142 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.142 INFO:teuthology.orchestra.run.smithi107.stdout:1972 writesame 0xa70b2fa thru 0xa70ec43 data_size 0x1(0x394a bytes) 2024-09-21T15:33:18.150 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:18.153 INFO:teuthology.orchestra.run.smithi107.stdout:1973 write 0x8a6f857 thru 0x8a76751 (0x6efb bytes) 2024-09-21T15:33:18.165 INFO:teuthology.orchestra.run.smithi107.stdout:1974 trunc from 0xd2890bb to 0x874f9c7 2024-09-21T15:33:18.181 INFO:teuthology.orchestra.run.smithi107.stdout:1976 read 0x3b46353 thru 0x3b4d22a (0x6ed8 bytes) 2024-09-21T15:33:18.190 INFO:teuthology.orchestra.run.smithi107.stdout:1977 write 0xa7e74b6 thru 0xa7e91dd (0x1d28 bytes) 2024-09-21T15:33:18.193 INFO:teuthology.orchestra.run.smithi107.stdout:1978 write 0x5c1b01e thru 0x5c25937 (0xa91a bytes) 2024-09-21T15:33:18.197 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.197 INFO:teuthology.orchestra.run.smithi107.stdout:1981 read 0x813d9f8 thru 0x81477ea (0x9df3 bytes) 2024-09-21T15:33:18.198 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.198 INFO:teuthology.orchestra.run.smithi107.stdout:1987 read 0xa0b2c69 thru 0xa0c158c (0xe924 bytes) 2024-09-21T15:33:18.198 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:18.207 INFO:teuthology.orchestra.run.smithi107.stdout:1991 writesame 0xc5af6fe thru 0xc5b4b7f data_size 0x1d(0x5482 bytes) 2024-09-21T15:33:18.207 INFO:teuthology.orchestra.run.smithi107.stdout:1992 write 0x154fd12 thru 0x155ee1e (0xf10d bytes) 2024-09-21T15:33:18.208 INFO:teuthology.orchestra.run.smithi107.stdout:1993 clone 15 order 22 su 1048576 sc 9 2024-09-21T15:33:18.239 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:18.239+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:18.313 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.314 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.327 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:18.371 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:18.371+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:18.439 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:18.435+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:18.492 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.492 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.506 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:18.555 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:18.555+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:18.630 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.630 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.641 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:18.639+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:18.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:18.741 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:18.739+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:18.777 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.777 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.791 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:18.876 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:18.875+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:18.943 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:18.939+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:18.956 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:18.956 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:18.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:19.026 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:33:19.029 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:19.029 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:19.029 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-21T15:33:19.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-21T15:33:19.044 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:19.043+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:19.142 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:19.142 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:19.145 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:19.143+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:19.158 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:19.210 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone14 from 0xc5b4b80 (overlap 0xa4639d) to 0x0 2024-09-21T15:33:19.246 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:19.243+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:19.280 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:19.280 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:19.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:19.345 INFO:tasks.ceph.mon.a.smithi107.stderr:2024-09-21T15:33:19.343+0000 7fbc162d8640 -1 mon.a@0(leader).osd e46 definitely_dead 0 2024-09-21T15:33:19.346 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:19.343+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:19.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:19.447 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:19.447+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:19.459 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T15:33:19.493 INFO:teuthology.orchestra.run.smithi107.stdout:1994 write 0xe78525f thru 0xe785914 (0x6b6 bytes) 2024-09-21T15:33:19.501 INFO:teuthology.orchestra.run.smithi107.stdout:1995 trunc from 0xe785915 to 0xb41fd41 2024-09-21T15:33:19.519 INFO:teuthology.orchestra.run.smithi107.stdout:1996 write 0x2de5406 thru 0x2df47a3 (0xf39e bytes) 2024-09-21T15:33:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:1997 read 0x82128b5 thru 0x822168b (0xedd7 bytes) 2024-09-21T15:33:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:2000 read 0x9307404 thru 0x930d72e (0x632b bytes) 2024-09-21T15:33:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:2001 punch from 0x57df1f2 to 0x57e8287, (0x9095 bytes) 2024-09-21T15:33:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:2002 write 0x397ea40 thru 0x398e999 (0xff5a bytes) 2024-09-21T15:33:19.530 INFO:teuthology.orchestra.run.smithi107.stdout:2003 punch from 0xa3633ce to 0xa364fc9, (0x1bfb bytes) 2024-09-21T15:33:19.531 INFO:teuthology.orchestra.run.smithi107.stdout:2004 write 0x5795393 thru 0x57a0c51 (0xb8bf bytes) 2024-09-21T15:33:19.537 INFO:teuthology.orchestra.run.smithi107.stdout:2007 writesame 0x6d6f77 thru 0x6dda1e data_size 0x4(0x6aa8 bytes) 2024-09-21T15:33:19.549 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:19.547+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:19.555 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:19.555 INFO:teuthology.orchestra.run.smithi107.stdout:2009 write 0x838fd15 thru 0x83991ed (0x94d9 bytes) 2024-09-21T15:33:19.561 INFO:teuthology.orchestra.run.smithi107.stdout:2010 punch from 0xb38dc27 to 0xb3974df, (0x98b8 bytes) 2024-09-21T15:33:19.561 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:19.561 INFO:teuthology.orchestra.run.smithi107.stdout:2014 read 0x38f157a thru 0x38f4e83 (0x390a bytes) 2024-09-21T15:33:19.575 INFO:teuthology.orchestra.run.smithi107.stdout:2015 trunc from 0xb41fd41 to 0x44b4f1f 2024-09-21T15:33:19.583 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:19.583 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:19.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:19.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:19.619 INFO:teuthology.orchestra.run.smithi107.stdout:2016 writesame 0x79c5c81 thru 0x79cb7e3 data_size 0x1(0x5b63 bytes) 2024-09-21T15:33:19.619 INFO:teuthology.orchestra.run.smithi107.stdout:2017 read 0x56c1fc6 thru 0x56c88d2 (0x690d bytes) 2024-09-21T15:33:19.619 INFO:teuthology.orchestra.run.smithi107.stdout:2018 read 0x3b2a364 thru 0x3b353a3 (0xb040 bytes) 2024-09-21T15:33:19.627 INFO:teuthology.orchestra.run.smithi107.stdout:2019 trunc from 0x79cb7e4 to 0xb10994a 2024-09-21T15:33:19.634 INFO:teuthology.orchestra.run.smithi107.stdout:2020 read 0x7b2a9e0 thru 0x7b2f0b4 (0x46d5 bytes) 2024-09-21T15:33:19.635 INFO:teuthology.orchestra.run.smithi107.stdout:2023 clone 16 order 19 su 131072 sc 10 2024-09-21T15:33:19.649 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:19.647+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:19.729 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:19.729 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:19.731 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:19.731 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:19.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:19.748 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:19.748 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:19.750 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:19.747+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:19.763 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:19.764 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:19.779 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:19.780 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:19.800 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:19.820 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:19.820 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:19.839 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:19.840 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:19.850 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:19.847+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:19.859 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:19.860 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:19.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:19.951 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:19.951+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:641 3.0 e26/21 3:04d455ab:::rbd_data.11911fd826fd.00000000000000a7:head v 26'8, mlcod=26'6)", 2024-09-21T15:33:20.042 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.564654887, 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001951022, 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474659+0000", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474657+0000", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474747+0000", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9559000000000001e-05 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474777+0000", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.991e-05 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474790+0000", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3246e-05 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474904+0000", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011387600000000001 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.474973+0000", 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8847e-05 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476526+0000", 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015536180000000001 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.476610+0000", 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3931999999999995e-05 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.495372550999999, 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:33:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:33:20.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3130 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'17, mlcod=28'14)", 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.215637433000001, 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018694930000000001, 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.051 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823677+0000", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823676+0000", 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823685+0000", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823711+0000", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6106e-05 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823726+0000", 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4644000000000001e-05 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823829+0000", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102505 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.823867+0000", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8606999999999998e-05 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825433+0000", 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015654810000000001 2024-09-21T15:33:20.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.825546+0000", 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113416 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.024456279999999, 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.055 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:33:20.059 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.014666188, 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:33:20.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.000712148000002, 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:33:20.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:33:20.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.992545003, 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:33:20.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.991798604, 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:33:20.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:33:20.076 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.973407628, 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:33:20.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:33:20.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.912541688000001, 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:33:20.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:33:20.084 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.900871351999999, 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7071 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'176, mlcod=35'173)", 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.133403421000001, 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0027688610000000001, 2024-09-21T15:33:20.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905911+0000", 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905910+0000", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905929+0000", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9409999999999999e-05 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905943+0000", 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4109e-05 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905950+0000", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.905995+0000", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4925000000000001e-05 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.906020+0000", 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4771000000000001e-05 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908648+0000", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002627622 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.908680+0000", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1795999999999998e-05 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7075 3.0 e35/21 3:04220828:::rbd_object_map.1191c706ae32:head v 35'178, mlcod=35'175)", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:20.093 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.127186822000001, 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030847729999999999, 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912127+0000", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912134+0000", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5449999999999998e-06 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912146+0000", 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1834999999999999e-05 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912153+0000", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7819999999999996e-06 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912186+0000", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3667999999999997e-05 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.912199+0000", 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2835e-05 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915184+0000", 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029846120000000002 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:05.915212+0000", 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8242000000000002e-05 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.097 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.942396254, 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.101 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.941485397999999, 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:20.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:33:20.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.940519471, 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:33:20.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:33:20.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7329 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'206, mlcod=36'204)", 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.93124605, 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003080213, 2024-09-21T15:33:20.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108068+0000", 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108067+0000", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108077+0000", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7990000000000005e-06 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108098+0000", 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1523999999999999e-05 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108109+0000", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0567999999999999e-05 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108153+0000", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4187999999999998e-05 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108175+0000", 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2089000000000001e-05 2024-09-21T15:33:20.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111116+0000", 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029405490000000002 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.111148+0000", 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2619000000000001e-05 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.114 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.930315489, 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:33:20.118 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:9864 3.9 e38/21 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head v 38'41, mlcod=37'39)", 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.5238509229999995, 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026427170000000002, 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515463+0000", 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.515462+0000", 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516535+0000", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001073084 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516551+0000", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5321000000000001e-05 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516560+0000", 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8259999999999999e-06 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516610+0000", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0844999999999999e-05 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.516625+0000", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518072+0000", 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001446578 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:10.518106+0000", 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4164e-05 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:10955 3.4 e39/21 3:21c0c826:::rbd_data.11917b2c111e.0000000000000016:head v 39'20, mlcod=38'18)", 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.3116039309999996, 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0020723389999999999, 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.123 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727710+0000", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727708+0000", 2024-09-21T15:33:20.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727720+0000", 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1742999999999999e-05 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727739+0000", 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8777e-05 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727748+0000", 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0299999999999999e-06 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727800+0000", 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2471e-05 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.727824+0000", 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3913999999999998e-05 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.729746+0000", 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0019218569999999999 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.729783+0000", 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6507000000000001e-05 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.127 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:20.128 INFO:teuthology.orchestra.run.smithi107.stderr:marked in osd.2. 2024-09-21T15:33:20.129 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:20.130 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-21T15:33:20.130 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:20.127+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:20.230 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:20.227+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:20.295 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:20.295 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:20.295 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:20.295 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:20.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:20.356 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:20.355+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:20.431 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:20.431 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:20.431 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:20.432 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:20.431+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:20.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:20.533 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:20.531+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:20.613 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.242806887, 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.214595711000001, 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:20.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:20.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:20.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:20.622 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.025296343000001, 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:20.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:20.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.893911652, 2024-09-21T15:33:20.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:20.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:20.632 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.732814248, 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.732761701000001, 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:20.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.732713575999998, 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:20.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:20.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.648 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.726459559999999, 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:20.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:20.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.653 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.720316688, 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:20.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:20.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:20.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:20.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.159543706000001, 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.664 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.092519339999999, 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:20.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.957399208999998, 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.670 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916758418000001, 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916737066, 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916723849, 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916711041999999, 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916670634999999, 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.916646167, 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.897424179000001, 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.810120596000001, 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:20.712 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:20.713 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:20.714 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:20.711+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:20.815 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:20.811+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:20.845 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:20.845 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:20.845 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:20.845 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:20.859 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:20.916 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:20.915+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:20.979 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:20.979 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:20.979 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:20.979 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:20.979 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:20.993 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:21.017 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:21.015+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:21.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:33:21.117 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:21.115+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.388680569000002, 2024-09-21T15:33:21.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00570851, 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732737+0000", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732760+0000", 2024-09-21T15:33:21.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2650000000000002e-05 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732808+0000", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8455999999999998e-05 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732829+0000", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0837000000000001e-05 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732965+0000", 2024-09-21T15:33:21.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013561899999999999 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733187+0000", 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022187800000000001 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733408+0000", 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022163799999999999 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.734741+0000", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013323199999999999 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738291+0000", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003550802 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738376+0000", 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4519999999999997e-05 2024-09-21T15:33:21.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738449+0000", 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2805999999999994e-05 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.445844833999999, 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:21.128 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:21.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.133 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:878 3.1a 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.124416727, 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072331519999999996, 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 878 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.134 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997002+0000", 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997018+0000", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6682999999999999e-05 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997047+0000", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8503000000000001e-05 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997067+0000", 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9962999999999998e-05 2024-09-21T15:33:21.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997172+0000", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000104676 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997378+0000", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000206465 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997965+0000", 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000586569 2024-09-21T15:33:21.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.999686+0000", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001720877 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004149+0000", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044633499999999996 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004192+0000", 2024-09-21T15:33:21.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3501000000000002e-05 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004237+0000", 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4848000000000002e-05 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.329747480999998, 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.139 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:21.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:21.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:21.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.293167366999999, 2024-09-21T15:33:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:21.147 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.274238672999999, 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:21.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:21.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1319 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.170632098999999, 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060901970000000003, 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950786+0000", 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950804+0000", 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8017e-05 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950836+0000", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1328999999999998e-05 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950855+0000", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9714000000000001e-05 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950976+0000", 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120635 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951033+0000", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7469999999999997e-05 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956831+0000", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057975910000000004 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956879+0000", 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7938000000000001e-05 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.155 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3381 3.9 3:91379144:::rbd_object_map.1191cae494be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.072589506, 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066017230000000003, 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3381 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048830+0000", 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:21.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048837+0000", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048850+0000", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2991e-05 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048858+0000", 2024-09-21T15:33:21.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049065+0000", 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020716899999999999 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049141+0000", 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6222000000000003e-05 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:21.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049454+0000", 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031307500000000001 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.050206+0000", 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00075177000000000002 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055362+0000", 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00515586 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055404+0000", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1974999999999997e-05 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055433+0000", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9254000000000001e-05 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:21.161 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.534489642, 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:21.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3986 3.9 3:931f7677:::rbd_data.119117b26f37.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.486111748999999, 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077269110000000004, 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3986 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635306+0000", 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635321+0000", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.431e-05 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635351+0000", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0633000000000003e-05 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635373+0000", 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.139e-05 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635463+0000", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599000000000005e-05 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635688+0000", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022439500000000001 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.637251+0000", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001563348 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642822+0000", 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055710539999999998 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642898+0000", 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5783000000000003e-05 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642982+0000", 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3814999999999996e-05 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.643036+0000", 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4187000000000001e-05 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.399267249000001, 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:21.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:21.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.396238519000001, 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:21.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:21.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:21.178 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4247 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'77, mlcod=31'75)", 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.285865004000001, 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069077940000000001, 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835554+0000", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835571+0000", 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835605+0000", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.375e-05 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835624+0000", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9606000000000002e-05 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835747+0000", 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122846 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835802+0000", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4703000000000001e-05 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842381+0000", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065796170000000003 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842464+0000", 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2169000000000006e-05 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.182 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4249 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'22, mlcod=31'20)", 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.285563603, 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006656276, 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835856+0000", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835873+0000", 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7397e-05 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835898+0000", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4995999999999999e-05 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835914+0000", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.562e-05 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836036+0000", 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121886 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836082+0000", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6430000000000001e-05 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842450+0000", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063680170000000001 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842513+0000", 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3377e-05 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.186 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.098033185999999, 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:21.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:21.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.047447549000001, 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:21.192 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:21.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9864 3.9 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.6060149, 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067473979999999999, 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9864 2024-09-21T15:33:21.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515403+0000", 2024-09-21T15:33:21.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516330+0000", 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00092668600000000002 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516362+0000", 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1402999999999997e-05 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516377+0000", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.523e-05 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516476+0000", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9205999999999999e-05 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516715+0000", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023897400000000001 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:21.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.517256+0000", 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054115999999999995 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.520752+0000", 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003495976 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.521884+0000", 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001132384 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522029+0000", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014489599999999999 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522153+0000", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000123936 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9922 3.1a 3:5f081d68:::rbd_data.1191f1341b63.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 10.491560557, 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060256199999999998, 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9922 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629858+0000", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629874+0000", 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5557e-05 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629902+0000", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8065999999999999e-05 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629916+0000", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630016+0000", 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9822999999999996e-05 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630166+0000", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015055900000000001 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.631290+0000", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001123474 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635698+0000", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044081859999999997 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635721+0000", 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096999999999999e-05 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635797+0000", 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6229000000000005e-05 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635886+0000", 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8578999999999993e-05 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.6857370839999994, 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11989 3.9 3:9126918e:::rbd_data.11917b2c111e.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.8151765969999998, 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00572151, 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11989 2024-09-21T15:33:21.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306242+0000", 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306255+0000", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.308e-05 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306288+0000", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2736e-05 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306309+0000", 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0939999999999999e-05 2024-09-21T15:33:21.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307487+0000", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011785350000000001 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307693+0000", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020588799999999999 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.308075+0000", 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038175999999999998 2024-09-21T15:33:21.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.310669+0000", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002594 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311745+0000", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001075398 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311865+0000", 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120353 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311966+0000", 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100846 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.217 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:21.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:21.219 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:21.215+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:21.319 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:21.315+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:21.387 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:21.387 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:21.387 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:21.387 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:21.402 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:21.420 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:21.419+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:21.521 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:21.519+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:21.522 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:21.523 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:21.523 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:21.537 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:21.623 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:21.623+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:21.708 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:21.708 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:21.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:21.708 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 44.919521392, 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:21.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:21.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:21.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:21.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.912910185999998, 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:21.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:21.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:21.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 29.338082467, 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.721 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:21.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:21.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:21.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:872 3.5 3:a31b1cfe:::rbd_data.11911fd826fd.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.71381002, 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0076432899999999996, 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 872 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993095+0000", 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:21.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993106+0000", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1588e-05 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993118+0000", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1914e-05 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993127+0000", 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7059999999999993e-06 2024-09-21T15:33:21.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993260+0000", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013325199999999999 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993527+0000", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000266464 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.994037+0000", 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051008300000000004 2024-09-21T15:33:21.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.996466+0000", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024291550000000001 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998600+0000", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021341989999999998 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998759+0000", 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000158269 2024-09-21T15:33:21.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000739+0000", 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0019806120000000001 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:989 3.c 3:35c0a0a7:::rbd_data.11911fd826fd.0000000000000037:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.669524447000001, 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007642383, 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.732 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 989 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037379+0000", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037392+0000", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2522e-05 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037411+0000", 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9052e-05 2024-09-21T15:33:21.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037424+0000", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2822e-05 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037498+0000", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3934000000000001e-05 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037639+0000", 2024-09-21T15:33:21.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014103499999999999 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037876+0000", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236782 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.039324+0000", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001448392 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044959+0000", 2024-09-21T15:33:21.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056346479999999999 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044994+0000", 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5728000000000001e-05 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045024+0000", 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9522e-05 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.036268474, 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:21.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:21.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.957451819999999, 2024-09-21T15:33:21.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.927098807, 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:21.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.919227112000002, 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.753 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:21.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.908047664000001, 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.758 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:21.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:21.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:21.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.902815298, 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:21.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.884330017, 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.757176515000001, 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.772 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:21.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.757020192999999, 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:21.777 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:21.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:21.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:21.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.751337968000001, 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.783 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:21.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:21.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.750375149, 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:21.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:21.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 13.989530225999999, 2024-09-21T15:33:21.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.797 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 13.680505591999999, 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:21.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.4014406150000003, 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:21.803 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:21.809 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.0706011179999999, 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:21.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:21.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:21.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:21.815 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:21.816 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:21.817 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:21.815+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:21.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:21.924 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:21.923+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:21.986 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:21.986 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:22.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:22.034 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:22.035 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:22.043 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:22.043+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:22.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:22.051 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:22.058 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:33:22.070 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:22.070 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:22.087 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:22.087 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:22.107 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:22.117 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:22.117 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:22.117 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:22.117 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:22.118 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:22.118 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:22.115+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:22.125 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:22.126 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:22.132 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:22.143 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:22.143 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:22.162 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:22.219 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:22.219+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:22.303 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:22.303 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:22.318 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:22.321 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:22.319+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:22.390 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone15 from 0xb10994a (overlap 0x44b4f1f) to 0x0 2024-09-21T15:33:22.423 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:22.419+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:22.494 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:22.495 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:22.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:22.522 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:22.519+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:22.623 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:22.619+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:22.634 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:22.635 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:22.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:22.651 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T15:33:22.663 INFO:teuthology.orchestra.run.smithi107.stdout:2025 trunc from 0xb10994a to 0x7ca2705 2024-09-21T15:33:22.706 INFO:teuthology.orchestra.run.smithi107.stdout:2026 write 0xa2260cc thru 0xa230117 (0xa04c bytes) 2024-09-21T15:33:22.713 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.713 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.714 INFO:teuthology.orchestra.run.smithi107.stdout:2030 punch from 0x7eb83 to 0x829b5, (0x3e32 bytes) 2024-09-21T15:33:22.714 INFO:teuthology.orchestra.run.smithi107.stdout:2031 read 0x95cfa10 thru 0x95dbb2c (0xc11d bytes) 2024-09-21T15:33:22.714 INFO:teuthology.orchestra.run.smithi107.stdout:2032 write 0x9c00422 thru 0x9c06fe4 (0x6bc3 bytes) 2024-09-21T15:33:22.724 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:22.723+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:22.734 INFO:teuthology.orchestra.run.smithi107.stdout:2033 write 0xb4e1a34 thru 0xb4e6165 (0x4732 bytes) 2024-09-21T15:33:22.742 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.742 INFO:teuthology.orchestra.run.smithi107.stdout:2036 read 0x83ac345 thru 0x83ac490 (0x14c bytes) 2024-09-21T15:33:22.742 INFO:teuthology.orchestra.run.smithi107.stdout:2037 read 0x818eefc thru 0x8197b2c (0x8c31 bytes) 2024-09-21T15:33:22.742 INFO:teuthology.orchestra.run.smithi107.stdout:2039 read 0x5f1a8c6 thru 0x5f23e79 (0x95b4 bytes) 2024-09-21T15:33:22.742 INFO:teuthology.orchestra.run.smithi107.stdout:2040 write 0x1718782 thru 0x1725031 (0xc8b0 bytes) 2024-09-21T15:33:22.752 INFO:teuthology.orchestra.run.smithi107.stdout:2041 read 0x7179eee thru 0x717b65d (0x1770 bytes) 2024-09-21T15:33:22.753 INFO:teuthology.orchestra.run.smithi107.stdout:2042 write 0x1a547a2 thru 0x1a57108 (0x2967 bytes) 2024-09-21T15:33:22.766 INFO:teuthology.orchestra.run.smithi107.stdout:2044 trunc from 0xb4e6166 to 0xd50313f 2024-09-21T15:33:22.777 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.777 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.778 INFO:teuthology.orchestra.run.smithi107.stdout:2049 read 0x33edd27 thru 0x33ee314 (0x5ee bytes) 2024-09-21T15:33:22.778 INFO:teuthology.orchestra.run.smithi107.stdout:2050 read 0x254fcff thru 0x255004c (0x34e bytes) 2024-09-21T15:33:22.780 INFO:teuthology.orchestra.run.smithi107.stdout:2052 read 0x15d1c6 thru 0x169286 (0xc0c1 bytes) 2024-09-21T15:33:22.780 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:22.780 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:22.781 INFO:teuthology.orchestra.run.smithi107.stdout:2053 writesame 0x8be5285 thru 0x8be7817 data_size 0x1(0x2593 bytes) 2024-09-21T15:33:22.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:22.798 INFO:teuthology.orchestra.run.smithi107.stdout:2054 read 0x4f131ad thru 0x4f22a8a (0xf8de bytes) 2024-09-21T15:33:22.814 INFO:teuthology.orchestra.run.smithi107.stdout:2055 write 0xebca870 thru 0xebd6ee0 (0xc671 bytes) 2024-09-21T15:33:22.819 INFO:teuthology.orchestra.run.smithi107.stdout:2056 read 0x1b17b20 thru 0x1b225f9 (0xaada bytes) 2024-09-21T15:33:22.820 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.820 INFO:teuthology.orchestra.run.smithi107.stdout:2059 read 0x88e868f thru 0x88f4a67 (0xc3d9 bytes) 2024-09-21T15:33:22.820 INFO:teuthology.orchestra.run.smithi107.stdout:2061 read 0x85f785c thru 0x85fd6b8 (0x5e5d bytes) 2024-09-21T15:33:22.820 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:22.824 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:22.823+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:22.836 INFO:teuthology.orchestra.run.smithi107.stdout:2065 trunc from 0xebd6ee1 to 0x87a31f6 2024-09-21T15:33:22.856 INFO:teuthology.orchestra.run.smithi107.stdout:2066 read 0x49c6b8b thru 0x49ceebf (0x8335 bytes) 2024-09-21T15:33:22.856 INFO:teuthology.orchestra.run.smithi107.stdout:2067 read 0x19b2f59 thru 0x19b4524 (0x15cc bytes) 2024-09-21T15:33:22.857 INFO:teuthology.orchestra.run.smithi107.stdout:2069 punch from 0xfad4a2 to 0xfb91b1, (0xbd0f bytes) 2024-09-21T15:33:22.857 INFO:teuthology.orchestra.run.smithi107.stdout:2071 read 0x38ca8d6 thru 0x38d5ead (0xb5d8 bytes) 2024-09-21T15:33:22.860 INFO:teuthology.orchestra.run.smithi107.stdout:2073 trunc from 0x87a31f6 to 0x6a55952 2024-09-21T15:33:22.906 INFO:teuthology.orchestra.run.smithi107.stdout:2075 trunc from 0x6a55952 to 0xd4865fb 2024-09-21T15:33:22.918 INFO:teuthology.orchestra.run.smithi107.stdout:2077 punch from 0x3c2c7e to 0x3cb230, (0x85b2 bytes) 2024-09-21T15:33:22.918 INFO:teuthology.orchestra.run.smithi107.stdout:2078 write 0x8a1e46 thru 0x8a5c88 (0x3e43 bytes) 2024-09-21T15:33:22.925 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:22.923+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:22.926 INFO:teuthology.orchestra.run.smithi107.stdout:2079 read 0x1e65ae8 thru 0x1e6e633 (0x8b4c bytes) 2024-09-21T15:33:22.927 INFO:teuthology.orchestra.run.smithi107.stdout:2081 read 0x72892c9 thru 0x72905b7 (0x72ef bytes) 2024-09-21T15:33:22.927 INFO:teuthology.orchestra.run.smithi107.stdout:2082 read 0x3aa3872 thru 0x3aa65fd (0x2d8c bytes) 2024-09-21T15:33:22.927 INFO:teuthology.orchestra.run.smithi107.stdout:2083 punch from 0xc48ac84 to 0xc49a16b, (0xf4e7 bytes) 2024-09-21T15:33:22.936 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:22.936 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:22.945 INFO:teuthology.orchestra.run.smithi107.stdout:2084 trunc from 0xd4865fb to 0x3e1939c 2024-09-21T15:33:22.958 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:22.994 INFO:teuthology.orchestra.run.smithi107.stdout:2085 read 0x2758ccd thru 0x275e343 (0x5677 bytes) 2024-09-21T15:33:23.026 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:23.023+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:23.028 INFO:teuthology.orchestra.run.smithi107.stdout:2087 write 0xe11de9e thru 0xe122352 (0x44b5 bytes) 2024-09-21T15:33:23.036 INFO:teuthology.orchestra.run.smithi107.stdout:2089 read 0x9ea4b97 thru 0x9eb3b25 (0xef8f bytes) 2024-09-21T15:33:23.036 INFO:teuthology.orchestra.run.smithi107.stdout:2090 write 0x99bdf67 thru 0x99c3f61 (0x5ffb bytes) 2024-09-21T15:33:23.044 INFO:teuthology.orchestra.run.smithi107.stdout:2091 writesame 0xcfefbde thru 0xcff1399 data_size 0xe(0x17bc bytes) 2024-09-21T15:33:23.050 INFO:teuthology.orchestra.run.smithi107.stdout:2092 read 0x30782d5 thru 0x307e54c (0x6278 bytes) 2024-09-21T15:33:23.051 INFO:teuthology.orchestra.run.smithi107.stdout:2093 write 0xc5f4879 thru 0xc5f5a4c (0x11d4 bytes) 2024-09-21T15:33:23.056 INFO:teuthology.orchestra.run.smithi107.stdout:2094 read 0xdccfb66 thru 0xdcd5901 (0x5d9c bytes) 2024-09-21T15:33:23.056 INFO:teuthology.orchestra.run.smithi107.stdout:2095 write 0x273e62e thru 0x2745dcc (0x779f bytes) 2024-09-21T15:33:23.066 INFO:teuthology.orchestra.run.smithi107.stdout:2096 read 0xc22f0e8 thru 0xc239db0 (0xacc9 bytes) 2024-09-21T15:33:23.066 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:23.066 INFO:teuthology.orchestra.run.smithi107.stdout:2098 read 0xa07b87f thru 0xa082b0a (0x728c bytes) 2024-09-21T15:33:23.072 INFO:teuthology.orchestra.run.smithi107.stdout:2099 trunc from 0xe122353 to 0xb11d465 2024-09-21T15:33:23.099 INFO:teuthology.orchestra.run.smithi107.stdout:2101 write 0x8bcb5cd thru 0x8bd55d5 (0xa009 bytes) 2024-09-21T15:33:23.104 INFO:teuthology.orchestra.run.smithi107.stdout:2102 read 0x539bf0b thru 0x53a2a41 (0x6b37 bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2103 writesame 0xbbbe3dd thru 0xbbc34d8 data_size 0x49(0x50fc bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2105 read 0x4957f1e thru 0x4967a4f (0xfb32 bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2107 punch from 0x22d0075 to 0x22d43e2, (0x436d bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2108 punch from 0x34a2afc to 0x34aed6d, (0xc271 bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2110 read 0x445bafe thru 0x445efcb (0x34ce bytes) 2024-09-21T15:33:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:2111 punch from 0x7a5598c to 0x7a5689a, (0xf0e bytes) 2024-09-21T15:33:23.127 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:23.123+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:23.132 INFO:teuthology.orchestra.run.smithi107.stdout:2112 write 0xd4ea18f thru 0xd4f0e21 (0x6c93 bytes) 2024-09-21T15:33:23.138 INFO:teuthology.orchestra.run.smithi107.stdout:2114 punch from 0x30ed550 to 0x30f1fd9, (0x4a89 bytes) 2024-09-21T15:33:23.141 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.141 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.142 INFO:teuthology.orchestra.run.smithi107.stdout:2115 trunc from 0xd4f0e22 to 0xbc417bf 2024-09-21T15:33:23.158 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2117 write 0xc2bfe71 thru 0xc2c29ea (0x2b7a bytes) 2024-09-21T15:33:23.182 INFO:teuthology.orchestra.run.smithi107.stdout:2118 punch from 0xb41c0db to 0xb41f790, (0x36b5 bytes) 2024-09-21T15:33:23.182 INFO:teuthology.orchestra.run.smithi107.stdout:2119 write 0xb445c19 thru 0xb44a689 (0x4a71 bytes) 2024-09-21T15:33:23.203 INFO:teuthology.orchestra.run.smithi107.stdout:2120 trunc from 0xc2c29eb to 0x2378406 2024-09-21T15:33:23.227 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:23.227+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:23.304 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.304 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:23.360 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:23.359+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:23.429 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:23.427+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:23.486 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.486 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:23.530 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:23.527+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:23.620 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.620 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.630 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:23.627+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:23.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:23.731 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:23.731+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:23.860 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:23.859+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:23.934 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:23.931+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:23.934 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:23.934 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:23.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:24.036 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:24.035+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:24.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:24.124 INFO:teuthology.orchestra.run.smithi107.stdout:2122 writesame 0xb4bb807 thru 0xb4be3e5 data_size 0x1(0x2bdf bytes) 2024-09-21T15:33:24.124 INFO:teuthology.orchestra.run.smithi107.stdout:2123 read 0x1babc3d thru 0x1bb550a (0x98ce bytes) 2024-09-21T15:33:24.132 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:24.132 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:24.135 INFO:teuthology.orchestra.run.smithi107.stdout:2125 trunc from 0xb4be3e6 to 0x51f89af 2024-09-21T15:33:24.136 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:24.135+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:24.149 INFO:teuthology.orchestra.run.smithi107.stdout:2126 read 0x112350c thru 0x11326db (0xf1d0 bytes) 2024-09-21T15:33:24.151 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:24.162 INFO:teuthology.orchestra.run.smithi107.stdout:2127 trunc from 0x51f89af to 0xd7fbc19 2024-09-21T15:33:24.172 INFO:teuthology.orchestra.run.smithi107.stdout:2128 punch from 0x60075a3 to 0x6010371, (0x8dce bytes) 2024-09-21T15:33:24.172 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.172 INFO:teuthology.orchestra.run.smithi107.stdout:2130 writesame 0x93d5145 thru 0x93de56f data_size 0x1(0x942b bytes) 2024-09-21T15:33:24.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:24.203 INFO:teuthology.orchestra.run.smithi107.stdout:2131 trunc from 0xd7fbc19 to 0xdb952c8 2024-09-21T15:33:24.231 INFO:teuthology.orchestra.run.smithi107.stdout:2132 trunc from 0xdb952c8 to 0x2428e76 2024-09-21T15:33:24.237 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:24.235+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:24.270 INFO:teuthology.orchestra.run.smithi107.stdout:2133 write 0x694eab4 thru 0x695db80 (0xf0cd bytes) 2024-09-21T15:33:24.293 INFO:teuthology.orchestra.run.smithi107.stdout:2134 write 0xc73c9eb thru 0xc73dc3d (0x1253 bytes) 2024-09-21T15:33:24.295 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:24.295 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:24.298 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.299 INFO:teuthology.orchestra.run.smithi107.stdout:2137 read 0x52709ff thru 0x5271f94 (0x1596 bytes) 2024-09-21T15:33:24.299 INFO:teuthology.orchestra.run.smithi107.stdout:2138 writesame 0xab434f1 thru 0xab50176 data_size 0x2f(0xcc86 bytes) 2024-09-21T15:33:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:2139 punch from 0xbabcb82 to 0xbacc4d8, (0xf956 bytes) 2024-09-21T15:33:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:2140 write 0xb6cd3d6 thru 0xb6dbb69 (0xe794 bytes) 2024-09-21T15:33:24.311 INFO:teuthology.orchestra.run.smithi107.stdout:2141 read 0x9b57dc0 thru 0x9b676c0 (0xf901 bytes) 2024-09-21T15:33:24.311 INFO:teuthology.orchestra.run.smithi107.stdout:2142 writesame 0xa377324 thru 0xa37eecb data_size 0xc(0x7ba8 bytes) 2024-09-21T15:33:24.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:2143 write 0xa75b633 thru 0xa75c4b9 (0xe87 bytes) 2024-09-21T15:33:24.323 INFO:teuthology.orchestra.run.smithi107.stdout:2144 writesame 0x7565afa thru 0x756bb16 data_size 0x5f(0x601d bytes) 2024-09-21T15:33:24.328 INFO:teuthology.orchestra.run.smithi107.stdout:2145 writesame 0x72b5a75 thru 0x72bf4ff data_size 0x1(0x9a8b bytes) 2024-09-21T15:33:24.338 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:24.335+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:24.359 INFO:teuthology.orchestra.run.smithi107.stdout:2146 read 0x4e8764c thru 0x4e8b684 (0x4039 bytes) 2024-09-21T15:33:24.359 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.359 INFO:teuthology.orchestra.run.smithi107.stdout:2148 read 0x5ada67c thru 0x5adbb9c (0x1521 bytes) 2024-09-21T15:33:24.359 INFO:teuthology.orchestra.run.smithi107.stdout:2149 write 0x5c6fd07 thru 0x5c7d6f6 (0xd9f0 bytes) 2024-09-21T15:33:24.363 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:24.364 INFO:teuthology.orchestra.run.smithi107.stdout:2150 write 0x861b44f thru 0x861ea2d (0x35df bytes) 2024-09-21T15:33:24.364 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:24.368 INFO:teuthology.orchestra.run.smithi107.stdout:2151 punch from 0x4299c24 to 0x42a4b3d, (0xaf19 bytes) 2024-09-21T15:33:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:2152 trunc from 0xc73dc3e to 0x7b737d4 2024-09-21T15:33:24.393 INFO:teuthology.orchestra.run.smithi107.stdout:2153 read 0x1d877e3 thru 0x1d8fff5 (0x8813 bytes) 2024-09-21T15:33:24.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:24.397 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:24.406 INFO:teuthology.orchestra.run.smithi107.stdout:2155 write 0x8a54f55 thru 0x8a57043 (0x20ef bytes) 2024-09-21T15:33:24.419 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:24.419 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:24.424 INFO:teuthology.orchestra.run.smithi107.stdout:2157 trunc from 0x8a57044 to 0x318922 2024-09-21T15:33:24.439 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:24.435+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:24.447 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:24.463 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:24.463 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:24.468 INFO:teuthology.orchestra.run.smithi107.stdout:2158 punch from 0x314015 to 0x316566, (0x2551 bytes) 2024-09-21T15:33:24.471 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:24.481 INFO:teuthology.orchestra.run.smithi107.stdout:2159 trunc from 0x318922 to 0x9a0ebc9 2024-09-21T15:33:24.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:24.494 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:24.494 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:24.494 INFO:teuthology.orchestra.run.smithi107.stdout:2160 read 0x68d055e thru 0x68dfeab (0xf94e bytes) 2024-09-21T15:33:24.495 INFO:teuthology.orchestra.run.smithi107.stdout:2162 trunc from 0x9a0ebc9 to 0x99a03e1 2024-09-21T15:33:24.506 INFO:teuthology.orchestra.run.smithi107.stdout:2165 trunc from 0x99a03e1 to 0xaf0f302 2024-09-21T15:33:24.515 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:24.515 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:24.516 INFO:teuthology.orchestra.run.smithi107.stdout:2166 read 0xad4d95f thru 0xad5bff7 (0xe699 bytes) 2024-09-21T15:33:24.516 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.518 INFO:teuthology.orchestra.run.smithi107.stdout:2168 trunc from 0xaf0f302 to 0x931b738 2024-09-21T15:33:24.540 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:24.539+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:24.541 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:24.542 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:24.547 INFO:teuthology.orchestra.run.smithi107.stdout:2170 write 0xeaae94c thru 0xeab2c09 (0x42be bytes) 2024-09-21T15:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout:2171 punch from 0x48acfdf to 0x48b16f4, (0x4715 bytes) 2024-09-21T15:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout:2172 read 0xcbe1142 thru 0xcbe20b1 (0xf70 bytes) 2024-09-21T15:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout:2176 read 0x4ef9013 thru 0x4f062ec (0xd2da bytes) 2024-09-21T15:33:24.553 INFO:teuthology.orchestra.run.smithi107.stdout:2179 write 0xdfe274 thru 0xe05a21 (0x77ae bytes) 2024-09-21T15:33:24.559 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.559 INFO:teuthology.orchestra.run.smithi107.stdout:2181 write 0xe81c5c7 thru 0xe82bea4 (0xf8de bytes) 2024-09-21T15:33:24.564 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.564 INFO:teuthology.orchestra.run.smithi107.stdout:2184 punch from 0x3b7c930 to 0x3b83290, (0x6960 bytes) 2024-09-21T15:33:24.564 INFO:teuthology.orchestra.run.smithi107.stdout:2185 write 0x8f0567f thru 0x8f0947c (0x3dfe bytes) 2024-09-21T15:33:24.569 INFO:teuthology.orchestra.run.smithi107.stdout:2186 write 0xa8855e2 thru 0xa892182 (0xcba1 bytes) 2024-09-21T15:33:24.574 INFO:teuthology.orchestra.run.smithi107.stdout:2187 write 0xd4f86c1 thru 0xd4ffd5c (0x769c bytes) 2024-09-21T15:33:24.579 INFO:teuthology.orchestra.run.smithi107.stdout:2188 read 0x1351ae0 thru 0x1353ed8 (0x23f9 bytes) 2024-09-21T15:33:24.579 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.589 INFO:teuthology.orchestra.run.smithi107.stdout:2190 trunc from 0xeab2c0a to 0x7c4880f 2024-09-21T15:33:24.608 INFO:teuthology.orchestra.run.smithi107.stdout:2193 writesame 0x20be4ee thru 0x20c7f24 data_size 0x1(0x9a37 bytes) 2024-09-21T15:33:24.618 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:24.620 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:24.620 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:24.620 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:24.620 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:24.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:24.640 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:24.639+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:24.672 INFO:teuthology.orchestra.run.smithi107.stdout:2195 writesame 0x8335cb4 thru 0x8340c54 data_size 0x7(0xafa1 bytes) 2024-09-21T15:33:24.690 INFO:teuthology.orchestra.run.smithi107.stdout:2196 write 0x9c9689b thru 0x9c99d74 (0x34da bytes) 2024-09-21T15:33:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:2198 write 0x6cd49a9 thru 0x6cd72fe (0x2956 bytes) 2024-09-21T15:33:24.700 INFO:teuthology.orchestra.run.smithi107.stdout:2199 write 0x4fb6941 thru 0x4fbfc74 (0x9334 bytes) 2024-09-21T15:33:24.708 INFO:teuthology.orchestra.run.smithi107.stdout:2200 trunc from 0x9c99d75 to 0xafc0708 2024-09-21T15:33:24.718 INFO:teuthology.orchestra.run.smithi107.stdout:2202 read 0xd6293e thru 0xd64e9e (0x2561 bytes) 2024-09-21T15:33:24.741 INFO:teuthology.orchestra.run.smithi107.stdout:2204 trunc from 0xafc0708 to 0xdbd2a0 2024-09-21T15:33:24.742 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:24.739+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:24.789 INFO:teuthology.orchestra.run.smithi107.stdout:2206 trunc from 0xdbd2a0 to 0xcb1efc3 2024-09-21T15:33:24.801 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:24.801 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.430392298999998, 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.802 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:24.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.402181122999998, 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:24.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:24.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.212881755000002, 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:24.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:24.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 31.081497064000001, 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:24.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.821 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.920399660000001, 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:24.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:24.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:24.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:24.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.920347112999998, 2024-09-21T15:33:24.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:24.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.920298987999999, 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:24.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:24.838 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.914044971999999, 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:24.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:24.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.907902100000001, 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:24.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.849 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.347129118000002, 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:24.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:24.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:24.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:24.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:24.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.280104752, 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:24.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:24.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:24.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.144984620999999, 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:24.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104343830000001, 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:24.870 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104322478, 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104309261000001, 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:24.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104296454, 2024-09-21T15:33:24.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.883 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104256047, 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:24.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:24.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.104231579, 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:24.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.085009590999999, 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:24.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:24.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:24.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.997706008000002, 2024-09-21T15:33:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:24.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:24.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:24.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:24.902 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.903 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:24.903 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:24.903 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:24.904 INFO:teuthology.orchestra.run.smithi107.stdout:2207 read 0x306b90d thru 0x30794fa (0xdbee bytes) 2024-09-21T15:33:24.905 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:24.905 INFO:teuthology.orchestra.run.smithi107.stdout:2208 trunc from 0xcb1efc3 to 0x370521e 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2209 read 0x17506f thru 0x17fc34 (0xabc6 bytes) 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2210 punch from 0x12fe727 to 0x1305683, (0x6f5c bytes) 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2211 punch from 0x21c1d34 to 0x21d092f, (0xebfb bytes) 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2212 trunc from 0x370521e to 0xd03a524 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2213 read 0x5da7b79 thru 0x5da7ffa (0x482 bytes) 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2216 punch from 0xb1b8176 to 0xb1c56e2, (0xd56c bytes) 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2218 trunc from 0xd03a524 to 0xd207ef3 2024-09-21T15:33:24.906 INFO:teuthology.orchestra.run.smithi107.stdout:2219 trunc from 0xd207ef3 to 0x630597a 2024-09-21T15:33:24.907 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:24.903+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:24.908 INFO:teuthology.orchestra.run.smithi107.stdout:2220 writesame 0x988095 thru 0x98c886 data_size 0x1(0x47f2 bytes) 2024-09-21T15:33:24.936 INFO:teuthology.orchestra.run.smithi107.stdout:2221 read 0x396aa9 thru 0x398658 (0x1bb0 bytes) 2024-09-21T15:33:24.940 INFO:teuthology.orchestra.run.smithi107.stdout:2222 trunc from 0x630597a to 0x770a52a 2024-09-21T15:33:24.962 INFO:teuthology.orchestra.run.smithi107.stdout:2223 write 0x7725833 thru 0x772c2a5 (0x6a73 bytes) 2024-09-21T15:33:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:2224 read 0x638fa2a thru 0x639e813 (0xedea bytes) 2024-09-21T15:33:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:2225 read 0x1540929 thru 0x154669c (0x5d74 bytes) 2024-09-21T15:33:24.985 INFO:teuthology.orchestra.run.smithi107.stdout:2227 trunc from 0x772c2a6 to 0x10ff31b 2024-09-21T15:33:25.005 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:25.003+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:25.033 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.033 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:25.033 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:25.033 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:25.035 INFO:teuthology.orchestra.run.smithi107.stdout:2228 write 0x6a4a56b thru 0x6a4e079 (0x3b0f bytes) 2024-09-21T15:33:25.042 INFO:teuthology.orchestra.run.smithi107.stdout:2229 punch from 0x1727827 to 0x172ca58, (0x5231 bytes) 2024-09-21T15:33:25.042 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.043 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.043 INFO:teuthology.orchestra.run.smithi107.stdout:2233 write 0x59323a6 thru 0x5935fa2 (0x3bfd bytes) 2024-09-21T15:33:25.048 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:25.049 INFO:teuthology.orchestra.run.smithi107.stdout:2235 punch from 0x50bdbee to 0x50c50d3, (0x74e5 bytes) 2024-09-21T15:33:25.049 INFO:teuthology.orchestra.run.smithi107.stdout:2237 write 0x276370b thru 0x276949b (0x5d91 bytes) 2024-09-21T15:33:25.080 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:33:25.084 INFO:teuthology.orchestra.run.smithi107.stdout:2238 writesame 0xcb3e2da thru 0xcb45b81 data_size 0x4e(0x78a8 bytes) 2024-09-21T15:33:25.084 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.085 INFO:teuthology.orchestra.run.smithi107.stdout:2240 read 0x77b510f thru 0x77b8aa0 (0x3992 bytes) 2024-09-21T15:33:25.085 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.085 INFO:teuthology.orchestra.run.smithi107.stdout:2243 read 0xc3238ea thru 0xc329fc2 (0x66d9 bytes) 2024-09-21T15:33:25.085 INFO:teuthology.orchestra.run.smithi107.stdout:2244 write 0xbd2a80c thru 0xbd2bb44 (0x1339 bytes) 2024-09-21T15:33:25.090 INFO:teuthology.orchestra.run.smithi107.stdout:2245 write 0x5ccc2ae thru 0x5cd4b1f (0x8872 bytes) 2024-09-21T15:33:25.095 INFO:teuthology.orchestra.run.smithi107.stdout:2246 punch from 0x489401b to 0x48964cb, (0x24b0 bytes) 2024-09-21T15:33:25.107 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:25.103+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:25.110 INFO:teuthology.orchestra.run.smithi107.stdout:2248 trunc from 0xcb45b82 to 0x4853e8b 2024-09-21T15:33:25.131 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:25.131 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:25.131 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T15:33:25.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-21T15:33:25.143 INFO:teuthology.orchestra.run.smithi107.stdout:2249 write 0xaac100f thru 0xaacb011 (0xa003 bytes) 2024-09-21T15:33:25.146 INFO:teuthology.orchestra.run.smithi107.stdout:2250 punch from 0x90bb0d3 to 0x90c5325, (0xa252 bytes) 2024-09-21T15:33:25.161 INFO:teuthology.orchestra.run.smithi107.stdout:2253 trunc from 0xaacb012 to 0xdb6e2d 2024-09-21T15:33:25.200 INFO:teuthology.orchestra.run.smithi107.stdout:2254 write 0xaef8fb2 thru 0xaeff784 (0x67d3 bytes) 2024-09-21T15:33:25.208 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:25.207+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:25.218 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:25.218 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:25.218 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:25.222 INFO:teuthology.orchestra.run.smithi107.stdout:2255 write 0xe6410a9 thru 0xe645946 (0x489e bytes) 2024-09-21T15:33:25.229 INFO:teuthology.orchestra.run.smithi107.stdout:2256 read 0x73f9448 thru 0x7408ec7 (0xfa80 bytes) 2024-09-21T15:33:25.233 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:25.234 INFO:teuthology.orchestra.run.smithi107.stdout:2258 trunc from 0xe645947 to 0xbe59e9d 2024-09-21T15:33:25.249 INFO:teuthology.orchestra.run.smithi107.stdout:2260 write 0xb8a528c thru 0xb8b25a8 (0xd31d bytes) 2024-09-21T15:33:25.254 INFO:teuthology.orchestra.run.smithi107.stdout:2261 punch from 0x568c76a to 0x56949bb, (0x8251 bytes) 2024-09-21T15:33:25.254 INFO:teuthology.orchestra.run.smithi107.stdout:2262 punch from 0x7b16fd6 to 0x7b22ac4, (0xbaee bytes) 2024-09-21T15:33:25.254 INFO:teuthology.orchestra.run.smithi107.stdout:2264 write 0xaf029e2 thru 0xaf03459 (0xa78 bytes) 2024-09-21T15:33:25.258 INFO:teuthology.orchestra.run.smithi107.stdout:2265 read 0x442314b thru 0x442c998 (0x984e bytes) 2024-09-21T15:33:25.258 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.258 INFO:teuthology.orchestra.run.smithi107.stdout:2268 write 0xf2eeb5 thru 0xf2f083 (0x1cf bytes) 2024-09-21T15:33:25.263 INFO:teuthology.orchestra.run.smithi107.stdout:2269 write 0x4b263a3 thru 0x4b2d1e9 (0x6e47 bytes) 2024-09-21T15:33:25.267 INFO:teuthology.orchestra.run.smithi107.stdout:2270 trunc from 0xbe59e9d to 0xb8eafe1 2024-09-21T15:33:25.277 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.277 INFO:teuthology.orchestra.run.smithi107.stdout:2272 read 0x4d5e5c4 thru 0x4d691b7 (0xabf4 bytes) 2024-09-21T15:33:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:2274 writesame 0xe408f61 thru 0xe40970d data_size 0x5(0x7ad bytes) 2024-09-21T15:33:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:2276 read 0x98d5a7e thru 0x98e253c (0xcabf bytes) 2024-09-21T15:33:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:2277 write 0x3c09c1c thru 0x3c0c3a3 (0x2788 bytes) 2024-09-21T15:33:25.291 INFO:teuthology.orchestra.run.smithi107.stdout:2278 writesame 0xbce6e65 thru 0xbcedfba data_size 0x2(0x7156 bytes) 2024-09-21T15:33:25.308 INFO:teuthology.orchestra.run.smithi107.stdout:2279 punch from 0xb6542de to 0xb65581e, (0x1540 bytes) 2024-09-21T15:33:25.308 INFO:teuthology.orchestra.run.smithi107.stdout:2280 writesame 0x63490e5 thru 0x634a76c data_size 0xe(0x1688 bytes) 2024-09-21T15:33:25.309 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:25.307+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:25.312 INFO:teuthology.orchestra.run.smithi107.stdout:2281 punch from 0xb43dd11 to 0xb447e9c, (0xa18b bytes) 2024-09-21T15:33:25.312 INFO:teuthology.orchestra.run.smithi107.stdout:2282 read 0xb2325b4 thru 0xb23ba9d (0x94ea bytes) 2024-09-21T15:33:25.324 INFO:teuthology.orchestra.run.smithi107.stdout:2284 trunc from 0xe40970e to 0x466fca7 2024-09-21T15:33:25.353 INFO:teuthology.orchestra.run.smithi107.stdout:2285 write 0x7b684b2 thru 0x7b6aced (0x283c bytes) 2024-09-21T15:33:25.357 INFO:teuthology.orchestra.run.smithi107.stdout:2287 writesame 0x28e4a12 thru 0x28f024c data_size 0x4f(0xb83b bytes) 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.627414109, 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00570851, 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732737+0000", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732760+0000", 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2650000000000002e-05 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732808+0000", 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8455999999999998e-05 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732829+0000", 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0837000000000001e-05 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732965+0000", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013561899999999999 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733187+0000", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022187800000000001 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733408+0000", 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022163799999999999 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.734741+0000", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013323199999999999 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738291+0000", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003550802 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738376+0000", 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4519999999999997e-05 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738449+0000", 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2805999999999994e-05 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.684578373999997, 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:25.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:25.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:25.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:25.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:878 3.1a 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.363150267000002, 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072331519999999996, 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.371 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 878 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997002+0000", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997018+0000", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6682999999999999e-05 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997047+0000", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8503000000000001e-05 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997067+0000", 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9962999999999998e-05 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997172+0000", 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000104676 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997378+0000", 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000206465 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997965+0000", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000586569 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.999686+0000", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001720877 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004149+0000", 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044633499999999996 2024-09-21T15:33:25.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004192+0000", 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3501000000000002e-05 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004237+0000", 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4848000000000002e-05 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.568481021, 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:25.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:25.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.531900907000001, 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.381 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:25.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:25.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:25.385 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.512972213000001, 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:25.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:25.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1319 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.409365639000001, 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060901970000000003, 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950786+0000", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950804+0000", 2024-09-21T15:33:25.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8017e-05 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950836+0000", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1328999999999998e-05 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950855+0000", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9714000000000001e-05 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950976+0000", 2024-09-21T15:33:25.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120635 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951033+0000", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7469999999999997e-05 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956831+0000", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057975910000000004 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956879+0000", 2024-09-21T15:33:25.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7938000000000001e-05 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3381 3.9 3:91379144:::rbd_object_map.1191cae494be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.311323045999998, 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066017230000000003, 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.394 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3381 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:25.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048830+0000", 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048837+0000", 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048850+0000", 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2991e-05 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048858+0000", 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049065+0000", 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020716899999999999 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049141+0000", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6222000000000003e-05 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049454+0000", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031307500000000001 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.050206+0000", 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00075177000000000002 2024-09-21T15:33:25.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055362+0000", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00515586 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055404+0000", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1974999999999997e-05 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055433+0000", 2024-09-21T15:33:25.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9254000000000001e-05 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.773223181999999, 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:25.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.404 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3986 3.9 3:931f7677:::rbd_data.119117b26f37.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.724845289000001, 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077269110000000004, 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3986 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:25.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635306+0000", 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635321+0000", 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.431e-05 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635351+0000", 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0633000000000003e-05 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635373+0000", 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.139e-05 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635463+0000", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599000000000005e-05 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635688+0000", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022439500000000001 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.637251+0000", 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001563348 2024-09-21T15:33:25.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642822+0000", 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055710539999999998 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642898+0000", 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5783000000000003e-05 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642982+0000", 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3814999999999996e-05 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.643036+0000", 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4187000000000001e-05 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:25.410 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.638000788999999, 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.414 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.634972058999999, 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:25.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4247 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'77, mlcod=31'75)", 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.524598544, 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069077940000000001, 2024-09-21T15:33:25.419 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835554+0000", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835571+0000", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835605+0000", 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.375e-05 2024-09-21T15:33:25.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835624+0000", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9606000000000002e-05 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835747+0000", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122846 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835802+0000", 2024-09-21T15:33:25.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4703000000000001e-05 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842381+0000", 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065796170000000003 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842464+0000", 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2169000000000006e-05 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4249 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'22, mlcod=31'20)", 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.524297142999998, 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006656276, 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835856+0000", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835873+0000", 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7397e-05 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835898+0000", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4995999999999999e-05 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835914+0000", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.562e-05 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836036+0000", 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121886 2024-09-21T15:33:25.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836082+0000", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6430000000000001e-05 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842450+0000", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063680170000000001 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842513+0000", 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3377e-05 2024-09-21T15:33:25.427 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.336766726, 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.428 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:25.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:25.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.286181088999999, 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:25.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9864 3.9 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.84474844, 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067473979999999999, 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9864 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.438 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515403+0000", 2024-09-21T15:33:25.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516330+0000", 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00092668600000000002 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516362+0000", 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1402999999999997e-05 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516377+0000", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.523e-05 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516476+0000", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9205999999999999e-05 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516715+0000", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023897400000000001 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:25.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.517256+0000", 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054115999999999995 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.520752+0000", 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003495976 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.521884+0000", 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001132384 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522029+0000", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014489599999999999 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522153+0000", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000123936 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9922 3.1a 3:5f081d68:::rbd_data.1191f1341b63.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:25.443 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.730294097, 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060256199999999998, 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9922 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629858+0000", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629874+0000", 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5557e-05 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629902+0000", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8065999999999999e-05 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629916+0000", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630016+0000", 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9822999999999996e-05 2024-09-21T15:33:25.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630166+0000", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015055900000000001 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.631290+0000", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001123474 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635698+0000", 2024-09-21T15:33:25.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044081859999999997 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635721+0000", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096999999999999e-05 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635797+0000", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6229000000000005e-05 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635886+0000", 2024-09-21T15:33:25.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8578999999999993e-05 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.924470624, 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.449 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:25.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.454 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11989 3.9 3:9126918e:::rbd_data.11917b2c111e.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.0539101370000008, 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00572151, 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11989 2024-09-21T15:33:25.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306242+0000", 2024-09-21T15:33:25.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306255+0000", 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.308e-05 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306288+0000", 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2736e-05 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306309+0000", 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0939999999999999e-05 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307487+0000", 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011785350000000001 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307693+0000", 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020588799999999999 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.308075+0000", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038175999999999998 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.310669+0000", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002594 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311745+0000", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001075398 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311865+0000", 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120353 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311966+0000", 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100846 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.460 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:25.462 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:25.463 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:25.459+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2289 write 0x3a5e262 thru 0x3a62f22 (0x4cc1 bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2290 write 0xbc77e22 thru 0xbc7c2fa (0x44d9 bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2291 writesame 0xa51a5d2 thru 0xa5216fb data_size 0x5(0x712a bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2292 read 0x555ad3b thru 0x555e7ed (0x3ab3 bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2297 punch from 0xad65382 to 0xad67c06, (0x2884 bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2301 writesame 0x5e2e21c thru 0x5e3c7ec data_size 0x9(0xe5d1 bytes) 2024-09-21T15:33:25.464 INFO:teuthology.orchestra.run.smithi107.stdout:2302 read 0xa313a97 thru 0xa31e61c (0xab86 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2303 punch from 0x1218406 to 0x121d70c, (0x5306 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2304 punch from 0x3ee2fe0 to 0x3ee9f95, (0x6fb5 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2305 punch from 0x153c248 to 0x1547c2b, (0xb9e3 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2306 punch from 0x8e1b5a4 to 0x8e2aea0, (0xf8fc bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2309 write 0xa5b319e thru 0xa5b85e8 (0x544b bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2311 punch from 0x86099fe to 0x860d634, (0x3c36 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2312 writesame 0x51d8094 thru 0x51e1982 data_size 0x31(0x98ef bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2313 read 0x7f8147a thru 0x7f8aac9 (0x9650 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2316 read 0x1cfa1d1 thru 0x1cfd7c2 (0x35f2 bytes) 2024-09-21T15:33:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:2317 trunc from 0xbc7c2fb to 0x54690a1 2024-09-21T15:33:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:2318 read 0x4063ca9 thru 0x406f1a3 (0xb4fb bytes) 2024-09-21T15:33:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:2321 write 0x4c4418c thru 0x4c4bcd8 (0x7b4d bytes) 2024-09-21T15:33:25.492 INFO:teuthology.orchestra.run.smithi107.stdout:2324 trunc from 0x54690a1 to 0xe0c48ba 2024-09-21T15:33:25.505 INFO:teuthology.orchestra.run.smithi107.stdout:2325 write 0x42c5d5e thru 0x42d0008 (0xa2ab bytes) 2024-09-21T15:33:25.511 INFO:teuthology.orchestra.run.smithi107.stdout:2326 writesame 0x44f58c2 thru 0x44fc323 data_size 0x33(0x6a62 bytes) 2024-09-21T15:33:25.518 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.532 INFO:teuthology.orchestra.run.smithi107.stdout:2328 trunc from 0xe0c48ba to 0x8a5b25e 2024-09-21T15:33:25.542 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:25.543 INFO:teuthology.orchestra.run.smithi107.stdout:2330 writesame 0x5628fdd thru 0x562df0d data_size 0x13(0x4f31 bytes) 2024-09-21T15:33:25.550 INFO:teuthology.orchestra.run.smithi107.stdout:2334 writesame 0x39fdbb7 thru 0x3a068b2 data_size 0xe(0x8cfc bytes) 2024-09-21T15:33:25.559 INFO:teuthology.orchestra.run.smithi107.stdout:2337 read 0x80582fa thru 0x8064eda (0xcbe1 bytes) 2024-09-21T15:33:25.563 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:25.559+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:25.567 INFO:teuthology.orchestra.run.smithi107.stdout:2338 trunc from 0x8a5b25e to 0x60f6094 2024-09-21T15:33:25.605 INFO:teuthology.orchestra.run.smithi107.stdout:2339 writesame 0xafc388c thru 0xafd0961 data_size 0x1(0xd0d6 bytes) 2024-09-21T15:33:25.616 INFO:teuthology.orchestra.run.smithi107.stdout:2341 trunc from 0xafd0962 to 0x72b9842 2024-09-21T15:33:25.629 INFO:teuthology.orchestra.run.smithi107.stdout:2342 read 0x5397fef thru 0x539e009 (0x601b bytes) 2024-09-21T15:33:25.629 INFO:teuthology.orchestra.run.smithi107.stdout:2343 writesame 0x4a1f75c thru 0x4a20741 data_size 0x25(0xfe6 bytes) 2024-09-21T15:33:25.640 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.640 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:25.640 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:25.640 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:25.655 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:25.659 INFO:teuthology.orchestra.run.smithi107.stdout:2345 write 0xa62969f thru 0xa62c2f3 (0x2c55 bytes) 2024-09-21T15:33:25.666 INFO:teuthology.orchestra.run.smithi107.stdout:2346 punch from 0x34202d5 to 0x342ccc5, (0xc9f0 bytes) 2024-09-21T15:33:25.666 INFO:teuthology.orchestra.run.smithi107.stdout:2347 punch from 0x325c1e7 to 0x3266f09, (0xad22 bytes) 2024-09-21T15:33:25.666 INFO:teuthology.orchestra.run.smithi107.stdout:2348 clone 17 order 20 su 524288 sc 7 2024-09-21T15:33:25.699 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:25.699+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:25.765 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:25.763+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:25.821 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.821 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:25.821 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:25.821 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:25.821 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:25.835 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:25.879 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:25.879+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:25.955 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:25.955 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.167019203999999, 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:25.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:25.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 47.160407997999997, 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.962 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.585580278999998, 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.968 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:25.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.973 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:872 3.5 3:a31b1cfe:::rbd_data.11911fd826fd.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.961307832000003, 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0076432899999999996, 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 872 2024-09-21T15:33:25.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993095+0000", 2024-09-21T15:33:25.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993106+0000", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1588e-05 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993118+0000", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1914e-05 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993127+0000", 2024-09-21T15:33:25.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7059999999999993e-06 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993260+0000", 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013325199999999999 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993527+0000", 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000266464 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:25.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.994037+0000", 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051008300000000004 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.996466+0000", 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024291550000000001 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998600+0000", 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021341989999999998 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998759+0000", 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000158269 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000739+0000", 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0019806120000000001 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.979 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:989 3.c 3:35c0a0a7:::rbd_data.11911fd826fd.0000000000000037:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.917022258999999, 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007642383, 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 989 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:25.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037379+0000", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037392+0000", 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2522e-05 2024-09-21T15:33:25.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037411+0000", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9052e-05 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037424+0000", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2822e-05 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037498+0000", 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3934000000000001e-05 2024-09-21T15:33:25.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037639+0000", 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014103499999999999 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037876+0000", 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236782 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.039324+0000", 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001448392 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044959+0000", 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056346479999999999 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044994+0000", 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5728000000000001e-05 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045024+0000", 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9522e-05 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.283766286000002, 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:25.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:25.990 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.204949632000002, 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.174596618999999, 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:25.999 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.166724924, 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.155545476, 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.005 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:26.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:26.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.150313109999999, 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:26.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:26.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.131827829000002, 2024-09-21T15:33:26.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.004674326999996, 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:26.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:26.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.004518005000001, 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:26.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:26.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:26.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.99883578, 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:26.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:26.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:26.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.997872960999999, 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.036 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:26.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:26.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:26.040 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.237028037999998, 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.928003404000002, 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:26.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:26.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.6489384270000009, 2024-09-21T15:33:26.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:26.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:26.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.3180989299999997, 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.063 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:26.065 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:26.063+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:moved 4 / 82 (4.87805%) 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 3.26917 -> 2.90474 (expected baseline 2.99479) 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.2 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.7 with 14 -> 13 pgs (1.36585 -> 1.26829 * mean) 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 3 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0004 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0005 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 weight 1.0000 -> 0.8144 2024-09-21T15:33:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 weight 1.0000 -> 0.8505 2024-09-21T15:33:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 weight 1.0000 -> 0.9086 2024-09-21T15:33:26.117 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-21T15:33:26.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-21T15:33:26.204 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:26.203+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:26.236 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:26.236 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:26.236 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:26.236 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:26.266 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:26.263+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:26.367 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:26.363+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:26.371 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:26.372 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:26.372 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:26.372 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:26.372 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:26.456 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 82 (0%) 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 2.90474 -> 2.90474 (expected baseline 2.99479) 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.6 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:oload 1000 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 3 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0004 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0045 2024-09-21T15:33:26.457 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:33:26.468 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:26.463+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:26.475 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T15:33:26.476 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-21T15:33:26.508 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:26.508 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:26.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:26.508 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.756148046, 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072729969999999998, 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751175+0000", 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751188+0000", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3774000000000001e-05 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751208+0000", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0173000000000002e-05 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751232+0000", 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3059999999999999e-05 2024-09-21T15:33:26.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751344+0000", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011212199999999999 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751354+0000", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9199999999999999e-06 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755656+0000", 2024-09-21T15:33:26.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043023430000000001 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755870+0000", 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000213809 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.756264+0000", 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039450599999999998 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.757715+0000", 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001451232 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758252+0000", 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053677 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758350+0000", 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7275000000000006e-05 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758450+0000", 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100603 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:982 3.10 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:26.515 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.472609859999999, 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0073293079999999997, 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 982 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034713+0000", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034725+0000", 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2439999999999999e-05 2024-09-21T15:33:26.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034761+0000", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6146000000000003e-05 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034781+0000", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9293999999999999e-05 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034849+0000", 2024-09-21T15:33:26.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7986999999999998e-05 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034960+0000", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011157 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.035187+0000", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022666599999999999 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.036202+0000", 2024-09-21T15:33:26.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001014741 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041899+0000", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056968909999999999 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041978+0000", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9405000000000003e-05 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.042045+0000", 2024-09-21T15:33:26.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6587999999999993e-05 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3615 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.495545794000002, 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060378640000000004, 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.521 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3615 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011776+0000", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011793+0000", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7116999999999999e-05 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011829+0000", 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5642999999999997e-05 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011850+0000", 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0780000000000001e-05 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011970+0000", 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120086 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:26.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011980+0000", 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0463e-05 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015470+0000", 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034891829999999999 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015592+0000", 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012278199999999999 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015898+0000", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00030570800000000002 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017316+0000", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001417602 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017684+0000", 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036833800000000001 2024-09-21T15:33:26.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017763+0000", 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9161999999999999e-05 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017817+0000", 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4147000000000002e-05 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.044010526000001, 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:26.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:26.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3898 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'14, mlcod=30'12)", 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.979630390000001, 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029235214999999998, 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:26.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527693+0000", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538136+0000", 2024-09-21T15:33:26.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010442932 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538184+0000", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8427000000000002e-05 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538203+0000", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9309000000000001e-05 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538310+0000", 2024-09-21T15:33:26.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010705300000000001 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538353+0000", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2636000000000003e-05 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.556853+0000", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018499548000000001 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.556930+0000", 2024-09-21T15:33:26.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7316000000000003e-05 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4246 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'21, mlcod=31'19)", 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.672075217, 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060841710000000002, 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835248+0000", 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835261+0000", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3027000000000001e-05 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835290+0000", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9096000000000001e-05 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835303+0000", 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3157e-05 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835425+0000", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121409 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835479+0000", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4490000000000002e-05 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.841236+0000", 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057565419999999999 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.841334+0000", 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8189999999999993e-05 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4248 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'22, mlcod=31'19)", 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.670177308, 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056396440000000001, 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.540 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837145+0000", 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837165+0000", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9786999999999999e-05 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837201+0000", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6006000000000001e-05 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837221+0000", 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.959e-05 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.838208+0000", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00098791500000000002 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.838266+0000", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7627000000000002e-05 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842683+0000", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044163930000000002 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842788+0000", 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105003 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4294 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'27, mlcod=31'25)", 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.583683101999998, 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012218685, 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923640+0000", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923653+0000", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2913e-05 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923693+0000", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9487000000000003e-05 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923712+0000", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9103e-05 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923881+0000", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016896500000000001 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923935+0000", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4570000000000001e-05 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935757+0000", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011821431 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935861+0000", 2024-09-21T15:33:26.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103936 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4296 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'28, mlcod=31'25)", 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.580577243, 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010318423, 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:26.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926746+0000", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926762+0000", 2024-09-21T15:33:26.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.609e-05 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926792+0000", 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9709999999999998e-05 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926809+0000", 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7367000000000002e-05 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.928929+0000", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021198329999999998 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.928980+0000", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1209999999999999e-05 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.936978+0000", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0079978110000000005 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937066+0000", 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8171999999999995e-05 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 15.813733150000001, 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:26.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:26.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:26.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:26.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:26.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:9956 3.a e38/21 ondisk, result = 0)", 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 15.773131437, 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029216085999999999, 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734192+0000", 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734203+0000", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0674e-05 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734229+0000", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6296e-05 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734243+0000", 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763385+0000", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029141469 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763396+0000", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1919999999999999e-05 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763410+0000", 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3346999999999999e-05 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.563 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 15.512362359000001, 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.564 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:26.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:10056 3.a e38/21 ondisk, result = 0)", 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:26.569 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 15.474933098999999, 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017158778, 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032390+0000", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032403+0000", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.296e-05 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032434+0000", 2024-09-21T15:33:26.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032448+0000", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3973e-05 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049337+0000", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016889676999999999 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049355+0000", 2024-09-21T15:33:26.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7370000000000001e-05 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049551+0000", 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019615799999999999 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.836848539, 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:26.573 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.578 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:10329 3.1e e38/34 ondisk, result = 0)", 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.798102266000001, 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018525979000000001, 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:26.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709221+0000", 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709231+0000", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8970000000000003e-06 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709257+0000", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.633e-05 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709273+0000", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.52e-05 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727598+0000", 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018325592000000002 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727611+0000", 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3035e-05 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727749+0000", 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013759199999999999 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10539 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.374767013, 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065352300000000004, 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10539 2024-09-21T15:33:26.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132555+0000", 2024-09-21T15:33:26.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132571+0000", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6024000000000001e-05 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132607+0000", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6112999999999998e-05 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132624+0000", 2024-09-21T15:33:26.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7093000000000001e-05 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132737+0000", 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011303600000000001 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132748+0000", 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1175999999999999e-05 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136310+0000", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0035614290000000001 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136448+0000", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000138839 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136820+0000", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037166800000000002 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.138255+0000", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001434472 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.138969+0000", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00071443799999999997 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.139041+0000", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2403000000000005e-05 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.139093+0000", 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1935999999999999e-05 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14228 3.a 3:517637f8:::rbd_data.1191bd191516.0000000000000010:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.5926524020000001, 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024412297999999999, 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14228 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914670+0000", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914685+0000", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4688999999999999e-05 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914719+0000", 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4115000000000001e-05 2024-09-21T15:33:26.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914734+0000", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5308999999999999e-05 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914876+0000", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000141936 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.915266+0000", 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039032900000000002 2024-09-21T15:33:26.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.919370+0000", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041030240000000003 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938983+0000", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019613087000000001 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939002+0000", 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8949000000000001e-05 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939050+0000", 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8664e-05 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939085+0000", 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4856e-05 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:14228 3.a e51/21 ondisk, result = 0)", 2024-09-21T15:33:26.594 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.575106414, 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067704619999999997, 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932217+0000", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932228+0000", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0852e-05 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932257+0000", 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9397e-05 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932272+0000", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4898e-05 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938957+0000", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066852090000000001 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938970+0000", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3115999999999999e-05 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938989+0000", 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8944e-05 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14394 3.a 3:56816452:::rbd_data.1191bd191516.0000000000000173:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.211579022, 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015816156000000001, 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:26.598 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14394 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295744+0000", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295755+0000", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.04e-05 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295776+0000", 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1183999999999999e-05 2024-09-21T15:33:26.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295785+0000", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1679999999999993e-06 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295908+0000", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122688 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.296137+0000", 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022960999999999999 2024-09-21T15:33:26.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.298904+0000", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027669909999999999 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311331+0000", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012426535000000001 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311411+0000", 2024-09-21T15:33:26.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0710000000000005e-05 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311498+0000", 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6823000000000005e-05 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311562+0000", 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4016000000000005e-05 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:14394 3.a e52/21 ondisk, result = 0)", 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2041542759999999, 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084053489999999995, 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303170+0000", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303177+0000", 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7970000000000003e-06 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303193+0000", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6563e-05 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303202+0000", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.456e-06 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311385+0000", 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0081836219999999998 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311400+0000", 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4649e-05 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311576+0000", 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017615600000000001 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:26.607 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:26.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:26.610 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:26.607+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:26.710 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:26.707+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:26.742 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:26.742 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:26.743 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:26.743 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.757 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:26.764 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:26.764 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:26.780 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:26.780 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:26.798 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:26.798 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:26.810 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:26.807+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:26.814 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:26.814 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:26.832 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:26.832 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:26.849 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:26.849 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:26.869 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:26.881 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:26.881 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:26.881 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:26.881 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:26.881 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:26.889 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:26.889 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:26.895 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:26.912 INFO:teuthology.orchestra.run.smithi107.stdout:moved 0 / 82 (0%) 2024-09-21T15:33:26.912 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:33:26.912 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 2.90474 -> 2.90474 (expected baseline 2.99479) 2024-09-21T15:33:26.912 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.6 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:33:26.912 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout:oload 1000 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 0.05 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0004 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0045 2024-09-21T15:33:26.913 INFO:teuthology.orchestra.run.smithi107.stderr:no change 2024-09-21T15:33:26.931 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T15:33:26.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-21T15:33:26.939 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:26.939+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:27.020 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:27.019+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:27.060 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.060 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:27.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:33:27.124 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:27.123+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:27.195 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.195 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:27.214 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:27.215+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:27.315 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:27.311+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:27.328 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.328 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:27.428 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:27.427+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:27.516 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:27.519+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:27.617 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:27.615+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:27.680 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.680 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.694 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:27.718 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:27.719+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:27.733 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:moved 2 / 82 (2.43902%) 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 2.90474 -> 2.90474 (expected baseline 2.99479) 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.6 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.0 with 13 -> 13 pgs (1.26829 -> 1.26829 * mean) 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:33:27.737 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 0.05 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0004 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0005 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 weight 0.8144 -> 0.7644 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 weight 0.8505 -> 0.8005 2024-09-21T15:33:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 weight 0.9086 -> 0.8586 2024-09-21T15:33:27.759 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T15:33:27.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-21T15:33:27.848 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:27.847+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:27.856 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:27.856 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:27.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:27.958 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone16 2024-09-21T15:33:27.958 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:27.955+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:28.020 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:28.019+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:28.055 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.055 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:28.122 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:28.118+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T15:33:28.148 INFO:teuthology.orchestra.run.smithi107.stdout:2350 write 0xbd84b5c thru 0xbd8b8c3 (0x6d68 bytes) 2024-09-21T15:33:28.153 INFO:teuthology.orchestra.run.smithi107.stdout:2351 writesame 0xa481347 thru 0xa48ab26 data_size 0x1e(0x97e0 bytes) 2024-09-21T15:33:28.158 INFO:teuthology.orchestra.run.smithi107.stdout:2352 read 0x39d0300 thru 0x39de152 (0xde53 bytes) 2024-09-21T15:33:28.158 INFO:teuthology.orchestra.run.smithi107.stdout:2353 read 0xbce7431 thru 0xbce9b15 (0x26e5 bytes) 2024-09-21T15:33:28.159 INFO:teuthology.orchestra.run.smithi107.stdout:2354 write 0x70cee61 thru 0x70dca80 (0xdc20 bytes) 2024-09-21T15:33:28.165 INFO:teuthology.orchestra.run.smithi107.stdout:2355 trunc from 0xbd8b8c4 to 0xa9901e5 2024-09-21T15:33:28.182 INFO:teuthology.orchestra.run.smithi107.stdout:2356 clone 18 order 19 su 262144 sc 9 2024-09-21T15:33:28.222 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:28.222+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:28.228 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.228 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:28.328 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:28.327+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:28.404 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.405 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:28.424 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:28.422+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:28.524 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:28.523+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.576 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:28.625 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:28.622+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:28.726 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:28.723+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:28.734 INFO:teuthology.orchestra.run.smithi107.stdout:moved 3 / 82 (3.65854%) 2024-09-21T15:33:28.734 INFO:teuthology.orchestra.run.smithi107.stdout:avg 10.25 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:stddev 2.90474 -> 2.72718 (expected baseline 2.99479) 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:min osd.6 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:max osd.0 with 13 -> 14 pgs (1.26829 -> 1.36585 * mean) 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:oload 110 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:max_change 0.05 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:max_change_osds 4 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:average_utilization 0.0004 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:overload_utilization 0.0005 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 weight 0.7644 -> 0.7144 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 weight 0.8005 -> 0.7505 2024-09-21T15:33:28.735 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 weight 0.8586 -> 0.8086 2024-09-21T15:33:28.766 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.766 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.782 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:28.868 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:28.867+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:28.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:28.928 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:28.926+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:28.937 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:28.938 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:28.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:29.029 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:29.026+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:29.044 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:29.044 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:29.062 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:29.062 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:29.078 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:29.078 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:29.094 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:29.094 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:29.106 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.737250967000001, 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:33:29.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:33:29.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.709039791000002, 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:33:29.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.519740423000002, 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:33:29.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:29.122 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 35.388355732000001, 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:33:29.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.126 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.227258327999998, 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.227205781000002, 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.132 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.138 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.227157656000003, 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:29.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3169 3.b 3:d327e917:::rbd_data.1191ab8bc3c6.0000000000000000:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.220903640000003, 2024-09-21T15:33:29.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011631401, 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3169 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883964+0000", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883962+0000", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883977+0000", 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.486e-05 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884011+0000", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4357000000000003e-05 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884032+0000", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1066000000000001e-05 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884128+0000", 2024-09-21T15:33:29.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5602999999999999e-05 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884255+0000", 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012714900000000001 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884568+0000", 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031346100000000001 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.893856+0000", 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0092876380000000008 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895490+0000", 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016345089999999999 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895550+0000", 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9691999999999997e-05 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895596+0000", 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5793e-05 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.214760767999998, 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:33:29.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:33:29.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.653987785999998, 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:33:29.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:33:29.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.58696342, 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.161 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:33:29.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.451843288999999, 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:33:29.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:33:29.172 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4178 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411202498000002, 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011619825, 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4178 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.173 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693666+0000", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693665+0000", 2024-09-21T15:33:29.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693668+0000", 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9569999999999998e-06 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693675+0000", 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:29.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693682+0000", 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.503e-06 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705098+0000", 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011415649999999999 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705174+0000", 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5525000000000006e-05 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705285+0000", 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011175699999999999 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.177 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411181146000001, 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:33:29.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:33:29.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411167929000001, 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:33:29.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:33:29.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411155122, 2024-09-21T15:33:29.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:33:29.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411114715, 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:33:29.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.411090247000001, 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.196 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.198 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:33:29.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:33:29.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.391868258999999, 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.204 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:33:29.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:33:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:33:29.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.213 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.304564675999998, 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:29.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:33:29.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:33:29.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:29.219 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:29.220 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:29.221 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:29.222 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:29.222 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:29.222 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:29.223 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:29.223 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:29.223 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:29.224 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:29.225 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:29.219+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:29.321 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:29.318+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:29.355 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:29.356 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:29.356 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:29.356 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:29.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:29.422 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:29.419+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:29.523 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:29.523+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:29.541 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:29.541 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:29.541 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:29.541 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:29.541 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:29.555 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:29.625 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:29.623+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:29.677 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:29.677 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:29.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:29.677 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:29.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.945059837999999, 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00570851, 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732740+0000", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732737+0000", 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:29.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732760+0000", 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2650000000000002e-05 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732808+0000", 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8455999999999998e-05 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732829+0000", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0837000000000001e-05 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.732965+0000", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013561899999999999 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733187+0000", 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022187800000000001 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.733408+0000", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022163799999999999 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.734741+0000", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0013323199999999999 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738291+0000", 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003550802 2024-09-21T15:33:29.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738376+0000", 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4519999999999997e-05 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.738449+0000", 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2805999999999994e-05 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.002224103000003, 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:878 3.1a 3:5b6c7cc7:::rbd_data.11911fd826fd.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:29.689 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.680795996000001, 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072331519999999996, 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 878 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997004+0000", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997002+0000", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997018+0000", 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6682999999999999e-05 2024-09-21T15:33:29.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997047+0000", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8503000000000001e-05 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997067+0000", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9962999999999998e-05 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997172+0000", 2024-09-21T15:33:29.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000104676 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997378+0000", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000206465 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.997965+0000", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000586569 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.999686+0000", 2024-09-21T15:33:29.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001720877 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004149+0000", 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044633499999999996 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004192+0000", 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3501000000000002e-05 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.004237+0000", 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4848000000000002e-05 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.886126750000003, 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.695 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:29.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:29.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.700 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.849546635999999, 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:29.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:29.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:29.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.830617942000003, 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:29.707 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:29.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.711 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1319 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.727011367999999, 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060901970000000003, 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950789+0000", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950786+0000", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950804+0000", 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8017e-05 2024-09-21T15:33:29.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950836+0000", 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1328999999999998e-05 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950855+0000", 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9714000000000001e-05 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950976+0000", 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120635 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.951033+0000", 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7469999999999997e-05 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956831+0000", 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057975910000000004 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956879+0000", 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7938000000000001e-05 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3381 3.9 3:91379144:::rbd_object_map.1191cae494be:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.628968774999997, 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066017230000000003, 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3381 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048831+0000", 2024-09-21T15:33:29.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048830+0000", 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048837+0000", 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048850+0000", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2991e-05 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.048858+0000", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7510000000000008e-06 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049065+0000", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020716899999999999 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049141+0000", 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6222000000000003e-05 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.049454+0000", 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031307500000000001 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.050206+0000", 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00075177000000000002 2024-09-21T15:33:29.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055362+0000", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00515586 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055404+0000", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1974999999999997e-05 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:57.055433+0000", 2024-09-21T15:33:29.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9254000000000001e-05 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.090868911000001, 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:29.722 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.726 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3986 3.9 3:931f7677:::rbd_data.119117b26f37.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.042491018, 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077269110000000004, 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3986 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:29.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635309+0000", 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635306+0000", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635321+0000", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.431e-05 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635351+0000", 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0633000000000003e-05 2024-09-21T15:33:29.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635373+0000", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.139e-05 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635463+0000", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599000000000005e-05 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.635688+0000", 2024-09-21T15:33:29.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022439500000000001 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.637251+0000", 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001563348 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642822+0000", 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055710539999999998 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642898+0000", 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5783000000000003e-05 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.642982+0000", 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3814999999999996e-05 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.643036+0000", 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4187000000000001e-05 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.955646518000002, 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:29.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:29.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:29.902 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.952617788000001, 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4247 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'77, mlcod=31'75)", 2024-09-21T15:33:29.907 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.842244272999999, 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069077940000000001, 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835556+0000", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835554+0000", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835571+0000", 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7166000000000001e-05 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835605+0000", 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.375e-05 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835624+0000", 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9606000000000002e-05 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835747+0000", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122846 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835802+0000", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4703000000000001e-05 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842381+0000", 2024-09-21T15:33:29.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065796170000000003 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842464+0000", 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2169000000000006e-05 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4249 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'22, mlcod=31'20)", 2024-09-21T15:33:29.912 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 30.841942872000001, 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006656276, 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835857+0000", 2024-09-21T15:33:29.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835856+0000", 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835873+0000", 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7397e-05 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835898+0000", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4995999999999999e-05 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835914+0000", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.562e-05 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836036+0000", 2024-09-21T15:33:29.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121886 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.836082+0000", 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6430000000000001e-05 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842450+0000", 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063680170000000001 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842513+0000", 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3377e-05 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.654412454999999, 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:29.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:29.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:29.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:29.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:29.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.603826818000002, 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:29.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:29.928 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9864 3.9 3:9267b98a:::rbd_data.1191f1341b63.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.162394168999999, 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067473979999999999, 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9864 2024-09-21T15:33:29.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515406+0000", 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.515403+0000", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516330+0000", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00092668600000000002 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516362+0000", 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1402999999999997e-05 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516377+0000", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.523e-05 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516476+0000", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9205999999999999e-05 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.516715+0000", 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023897400000000001 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.517256+0000", 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00054115999999999995 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.520752+0000", 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003495976 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.521884+0000", 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001132384 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522029+0000", 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014489599999999999 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.522153+0000", 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000123936 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9922 3.1a 3:5f081d68:::rbd_data.1191f1341b63.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.047939826, 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060256199999999998, 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9922 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629860+0000", 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629858+0000", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629874+0000", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5557e-05 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629902+0000", 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8065999999999999e-05 2024-09-21T15:33:29.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.629916+0000", 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630016+0000", 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9822999999999996e-05 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.630166+0000", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015055900000000001 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.631290+0000", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001123474 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635698+0000", 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044081859999999997 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635721+0000", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096999999999999e-05 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635797+0000", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6229000000000005e-05 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.635886+0000", 2024-09-21T15:33:29.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8578999999999993e-05 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.242116353, 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:29.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:29.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:29.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11989 3.9 3:9126918e:::rbd_data.11917b2c111e.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 13.371555866, 2024-09-21T15:33:29.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00572151, 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11989 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:29.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306244+0000", 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306242+0000", 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306255+0000", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.308e-05 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306288+0000", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2736e-05 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.306309+0000", 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0939999999999999e-05 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307487+0000", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011785350000000001 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.307693+0000", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020588799999999999 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.308075+0000", 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038175999999999998 2024-09-21T15:33:29.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.310669+0000", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002594 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311745+0000", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001075398 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311865+0000", 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120353 2024-09-21T15:33:29.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311966+0000", 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100846 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:29.953 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:29.954 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:33:29.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:29.956 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:29.951+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:30.056 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:30.055+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:30.089 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:30.089 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:30.089 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:30.089 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:30.103 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:30.156 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:30.154+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:30.225 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:30.226 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:30.226 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:30.226 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:30.226 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:30.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:30.284 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:30.282+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:30.359 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:30.355+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:30.413 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.623994680000003, 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:30.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:30.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:30.420 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.617383474, 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:30.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:30.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 38.042555755000002, 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.427 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:30.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:30.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:872 3.5 3:a31b1cfe:::rbd_data.11911fd826fd.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.418283307999999, 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0076432899999999996, 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 872 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993095+0000", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993106+0000", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1588e-05 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993118+0000", 2024-09-21T15:33:30.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1914e-05 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993127+0000", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7059999999999993e-06 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993260+0000", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013325199999999999 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993527+0000", 2024-09-21T15:33:30.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000266464 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.994037+0000", 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051008300000000004 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.996466+0000", 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024291550000000001 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998600+0000", 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021341989999999998 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998759+0000", 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000158269 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000739+0000", 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0019806120000000001 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.438 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:989 3.c 3:35c0a0a7:::rbd_data.11911fd826fd.0000000000000037:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.373997735000003, 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007642383, 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 989 2024-09-21T15:33:30.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037379+0000", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037392+0000", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2522e-05 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037411+0000", 2024-09-21T15:33:30.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9052e-05 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037424+0000", 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2822e-05 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037498+0000", 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3934000000000001e-05 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037639+0000", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014103499999999999 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037876+0000", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236782 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.039324+0000", 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001448392 2024-09-21T15:33:30.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044959+0000", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056346479999999999 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044994+0000", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5728000000000001e-05 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045024+0000", 2024-09-21T15:33:30.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9522e-05 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.740741761999999, 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.445 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:30.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:30.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:30.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:30.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.661925107999998, 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:30.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:30.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.631572095000003, 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.457 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.623700399999997, 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.463 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:30.468 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.612520951999997, 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.469 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:30.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.607288586000003, 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.475 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:30.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.479 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.588803304999999, 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:30.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.483 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.461649803, 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.484 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:30.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:30.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.489 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.461493480999998, 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.490 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:30.495 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.455811255999997, 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:30.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:30.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:30.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.454848437000003, 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.502 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.694003513999998, 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.507 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.512 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.384978879999998, 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.513 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.518 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.105913902999999, 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:30.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:30.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.525 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 13.775074406, 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.526 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:30.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:30.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:30.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:30.531 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:30.533 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:30.533 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:30.534+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:30.634 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:30.631+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:30.701 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:30.701 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:30.716 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:30.750 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:33:30.763 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:30.762+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:30.808 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone17 from 0xa9901e5 (overlap 0xa62c2f4) to 0x0 2024-09-21T15:33:30.836 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:30.835+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:30.863 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:30.863 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:30.863 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:30.863 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:30.863 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:30.884 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:30.971 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:30.970+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:31.038 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:31.035+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12 3.a 3:56f28e0f:::rbd_header.1191d115b710:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.302397356999997, 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0072729969999999998, 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.055 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751177+0000", 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751175+0000", 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751188+0000", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3774000000000001e-05 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751208+0000", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0173000000000002e-05 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751232+0000", 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3059999999999999e-05 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751344+0000", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011212199999999999 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.751354+0000", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9199999999999999e-06 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755656+0000", 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043023430000000001 2024-09-21T15:33:31.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.755870+0000", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000213809 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.756264+0000", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039450599999999998 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.757715+0000", 2024-09-21T15:33:31.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001451232 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758252+0000", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053677 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758350+0000", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7275000000000006e-05 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:48.758450+0000", 2024-09-21T15:33:31.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100603 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:982 3.10 3:0d5b9d93:::rbd_data.11911fd826fd.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 38.018859171000003, 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0073293079999999997, 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 982 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034715+0000", 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034713+0000", 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034725+0000", 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2439999999999999e-05 2024-09-21T15:33:31.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034761+0000", 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6146000000000003e-05 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034781+0000", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9293999999999999e-05 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034849+0000", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7986999999999998e-05 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.034960+0000", 2024-09-21T15:33:31.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011157 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.035187+0000", 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022666599999999999 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.036202+0000", 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001014741 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041899+0000", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056968909999999999 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.041978+0000", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9405000000000003e-05 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.042045+0000", 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6587999999999993e-05 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.068 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3615 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.041795104999999, 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060378640000000004, 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3615 2024-09-21T15:33:31.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011779+0000", 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011776+0000", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011793+0000", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7116999999999999e-05 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011829+0000", 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5642999999999997e-05 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011850+0000", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0780000000000001e-05 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011970+0000", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120086 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.011980+0000", 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0463e-05 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015470+0000", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034891829999999999 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015592+0000", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012278199999999999 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.015898+0000", 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00030570800000000002 2024-09-21T15:33:31.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017316+0000", 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001417602 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017684+0000", 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036833800000000001 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017763+0000", 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9161999999999999e-05 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.017817+0000", 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4147000000000002e-05 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.075 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.590259836999998, 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.079 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3898 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'14, mlcod=30'12)", 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.525879701000001, 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029235214999999998, 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:31.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527695+0000", 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.527693+0000", 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538136+0000", 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010442932 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538184+0000", 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8427000000000002e-05 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538203+0000", 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9309000000000001e-05 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538310+0000", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010705300000000001 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.538353+0000", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2636000000000003e-05 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.556853+0000", 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018499548000000001 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.556930+0000", 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7316000000000003e-05 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4246 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'21, mlcod=31'19)", 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.218324527999997, 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060841710000000002, 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.084 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835250+0000", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835248+0000", 2024-09-21T15:33:31.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835261+0000", 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3027000000000001e-05 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835290+0000", 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9096000000000001e-05 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835303+0000", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3157e-05 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835425+0000", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121409 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.835479+0000", 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4490000000000002e-05 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.841236+0000", 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0057565419999999999 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.841334+0000", 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8189999999999993e-05 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4248 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'22, mlcod=31'19)", 2024-09-21T15:33:31.088 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.216426619000003, 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056396440000000001, 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837148+0000", 2024-09-21T15:33:31.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837145+0000", 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837165+0000", 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9786999999999999e-05 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837201+0000", 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6006000000000001e-05 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.837221+0000", 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.959e-05 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.838208+0000", 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00098791500000000002 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.838266+0000", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7627000000000002e-05 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842683+0000", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0044163930000000002 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.842788+0000", 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105003 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.092 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4294 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'27, mlcod=31'25)", 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.129932412999999, 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012218685, 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923642+0000", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923640+0000", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923653+0000", 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2913e-05 2024-09-21T15:33:31.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923693+0000", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9487000000000003e-05 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923712+0000", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9103e-05 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923881+0000", 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016896500000000001 2024-09-21T15:33:31.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.923935+0000", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4570000000000001e-05 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935757+0000", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011821431 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.935861+0000", 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103936 2024-09-21T15:33:31.096 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4296 3.1e e31/21 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head v 31'28, mlcod=31'25)", 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.126826553999997, 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010318423, 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926748+0000", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926746+0000", 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926762+0000", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.609e-05 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926792+0000", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9709999999999998e-05 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.926809+0000", 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7367000000000002e-05 2024-09-21T15:33:31.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.928929+0000", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021198329999999998 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.928980+0000", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1209999999999999e-05 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.936978+0000", 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0079978110000000005 2024-09-21T15:33:31.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.937066+0000", 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8171999999999995e-05 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.359982461000001, 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:31.101 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.102 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.106 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:9956 3.a e38/21 ondisk, result = 0)", 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.319380748, 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029216085999999999, 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734194+0000", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734192+0000", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734203+0000", 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0674e-05 2024-09-21T15:33:31.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734229+0000", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6296e-05 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.734243+0000", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.414e-05 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763385+0000", 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029141469 2024-09-21T15:33:31.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763396+0000", 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1919999999999999e-05 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763410+0000", 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3346999999999999e-05 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.058611670000001, 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:31.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:10056 3.a e38/21 ondisk, result = 0)", 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:31.116 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.021182410000002, 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017158778, 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032392+0000", 2024-09-21T15:33:31.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032390+0000", 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032403+0000", 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.296e-05 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032434+0000", 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.032448+0000", 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3973e-05 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049337+0000", 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016889676999999999 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049355+0000", 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7370000000000001e-05 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049551+0000", 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019615799999999999 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.120 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.383097849999999, 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:31.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:31.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.126 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:10329 3.1e e38/34 ondisk, result = 0)", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.344351577000001, 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018525979000000001, 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709223+0000", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709221+0000", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709231+0000", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8970000000000003e-06 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709257+0000", 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.633e-05 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.709273+0000", 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.52e-05 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727598+0000", 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018325592000000002 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727611+0000", 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3035e-05 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727749+0000", 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013759199999999999 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.130 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10539 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.921016324, 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065352300000000004, 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10539 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132558+0000", 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132555+0000", 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132571+0000", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6024000000000001e-05 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132607+0000", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6112999999999998e-05 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132624+0000", 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7093000000000001e-05 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132737+0000", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011303600000000001 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.132748+0000", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1175999999999999e-05 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136310+0000", 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0035614290000000001 2024-09-21T15:33:31.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136448+0000", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000138839 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.136820+0000", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037166800000000002 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.138255+0000", 2024-09-21T15:33:31.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001434472 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.138969+0000", 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00071443799999999997 2024-09-21T15:33:31.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.139041+0000", 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2403000000000005e-05 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.139093+0000", 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1935999999999999e-05 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14228 3.a 3:517637f8:::rbd_data.1191bd191516.0000000000000010:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.1389017130000001, 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024412297999999999, 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14228 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914673+0000", 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914670+0000", 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914685+0000", 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4688999999999999e-05 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914719+0000", 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4115000000000001e-05 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914734+0000", 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5308999999999999e-05 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.914876+0000", 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000141936 2024-09-21T15:33:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.915266+0000", 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00039032900000000002 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.919370+0000", 2024-09-21T15:33:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041030240000000003 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938983+0000", 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019613087000000001 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939002+0000", 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8949000000000001e-05 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939050+0000", 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8664e-05 2024-09-21T15:33:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.939085+0000", 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4856e-05 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:14228 3.a e51/21 ondisk, result = 0)", 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.1213557249999999, 2024-09-21T15:33:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067704619999999997, 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932219+0000", 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932217+0000", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932228+0000", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0852e-05 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932257+0000", 2024-09-21T15:33:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9397e-05 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.932272+0000", 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4898e-05 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938957+0000", 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066852090000000001 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938970+0000", 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3115999999999999e-05 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.938989+0000", 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8944e-05 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14394 3.a 3:56816452:::rbd_data.1191bd191516.0000000000000173:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.757828333, 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015816156000000001, 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14394 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295746+0000", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295744+0000", 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295755+0000", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.04e-05 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295776+0000", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1183999999999999e-05 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295785+0000", 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1679999999999993e-06 2024-09-21T15:33:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.295908+0000", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122688 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.296137+0000", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022960999999999999 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.298904+0000", 2024-09-21T15:33:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027669909999999999 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311331+0000", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012426535000000001 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311411+0000", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0710000000000005e-05 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311498+0000", 2024-09-21T15:33:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6823000000000005e-05 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311562+0000", 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4016000000000005e-05 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:14394 3.a e52/21 ondisk, result = 0)", 2024-09-21T15:33:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.7504035870000001, 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084053489999999995, 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303171+0000", 2024-09-21T15:33:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303170+0000", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303177+0000", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7970000000000003e-06 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303193+0000", 2024-09-21T15:33:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6563e-05 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.303202+0000", 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.456e-06 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311385+0000", 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0081836219999999998 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311400+0000", 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4649e-05 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:25.311576+0000", 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017615600000000001 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:31.162 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:31.164 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:31.164 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:31.159+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:31.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:31.273 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:31.271+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:31.334 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:31.334 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:31.334 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:31.334 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:31.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:31.378 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:31.379 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:31.394 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:31.396 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:31.394+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:31.410 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:31.429 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:31.429 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:31.449 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:31.467 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:31.462+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:31.468 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:31.469 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:31.471 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:31.472 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:31.486 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:31.489 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:31.489 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:31.510 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:31.511 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:31.576 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:31.575+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:31.656 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:31.656 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:31.669 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:31.667+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:31.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:31.769 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:31.766+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:31.807 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:31.807 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:31.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:31.870 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:31.867+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:31.943 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:31.943 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:31.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:31.971 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:31.970+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:32.072 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:32.071+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:32.118 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:32.118 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:32.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:32.176 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:32.175+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:32.274 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:32.271+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:32.294 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:32.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:32.396 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:32.395+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:32.476 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:32.474+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:32.478 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:32.478 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:32.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:32.580 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:32.579+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:32.653 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:32.653 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:32.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:32.677 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:32.674+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:32.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:33:32.777 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:32.775+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:32.809 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T15:33:32.843 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:32.843 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:32.862 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:32.878 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:32.874+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:32.979 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:32.974+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:33.022 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:33.022 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:33.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:33.080 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:33.078+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:33.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T15:33:33.107+0000 7f161cbea640 0 --2- 172.21.15.107:0/85506974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1618141320 0x7f161813d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T15:33:33.113 INFO:teuthology.orchestra.run.smithi107.stdout:2357 writesame 0x752c3b thru 0x75f84b data_size 0x11(0xcc11 bytes) 2024-09-21T15:33:33.123 INFO:teuthology.orchestra.run.smithi107.stdout:2358 read 0x58147da thru 0x58195f1 (0x4e18 bytes) 2024-09-21T15:33:33.124 INFO:teuthology.orchestra.run.smithi107.stdout:2359 read 0x605759e thru 0x6065752 (0xe1b5 bytes) 2024-09-21T15:33:33.124 INFO:teuthology.orchestra.run.smithi107.stdout:2360 writesame 0x51cf7f2 thru 0x51cfe00 data_size 0xb(0x60f bytes) 2024-09-21T15:33:33.132 INFO:teuthology.orchestra.run.smithi107.stdout:2361 read 0x6ae16b8 thru 0x6ae47ed (0x3136 bytes) 2024-09-21T15:33:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:2362 writesame 0xc3ce069 thru 0xc3cecff data_size 0x1(0xc97 bytes) 2024-09-21T15:33:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:2363 read 0x3b5fb1b thru 0x3b6ab13 (0xaff9 bytes) 2024-09-21T15:33:33.142 INFO:teuthology.orchestra.run.smithi107.stdout:2364 punch from 0x4db85ef to 0x4dc16b6, (0x90c7 bytes) 2024-09-21T15:33:33.156 INFO:teuthology.orchestra.run.smithi107.stdout:2367 trunc from 0xc3ced00 to 0x23c575b 2024-09-21T15:33:33.166 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:33.167 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:33.180 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:33.179+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:33.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:33.199 INFO:teuthology.orchestra.run.smithi107.stdout:2370 write 0xd3e6c93 thru 0xd3f1c40 (0xafae bytes) 2024-09-21T15:33:33.203 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.203 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.210 INFO:teuthology.orchestra.run.smithi107.stdout:2374 trunc from 0xd3f1c41 to 0x8187d4d 2024-09-21T15:33:33.233 INFO:teuthology.orchestra.run.smithi107.stdout:2375 trunc from 0x8187d4d to 0xe1c181 2024-09-21T15:33:33.279 INFO:teuthology.orchestra.run.smithi107.stdout:2376 write 0xbfe1a66 thru 0xbfe1ece (0x469 bytes) 2024-09-21T15:33:33.280 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:33.279+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:33.284 INFO:teuthology.orchestra.run.smithi107.stdout:2377 writesame 0xa1e31a2 thru 0xa1effa7 data_size 0x1(0xce06 bytes) 2024-09-21T15:33:33.326 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:33.326 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:33.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:33.354 INFO:teuthology.orchestra.run.smithi107.stdout:2378 punch from 0x63d998c to 0x63dac63, (0x12d7 bytes) 2024-09-21T15:33:33.354 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.355 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.355 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.355 INFO:teuthology.orchestra.run.smithi107.stdout:2383 write 0x9d55c5d thru 0x9d5891e (0x2cc2 bytes) 2024-09-21T15:33:33.369 INFO:teuthology.orchestra.run.smithi107.stdout:2384 trunc from 0xbfe1ecf to 0x7407d84 2024-09-21T15:33:33.381 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:33.378+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:33.383 INFO:teuthology.orchestra.run.smithi107.stdout:2385 punch from 0x5e80bac to 0x5e89e94, (0x92e8 bytes) 2024-09-21T15:33:33.383 INFO:teuthology.orchestra.run.smithi107.stdout:2386 punch from 0x35a3d4e to 0x35ab0da, (0x738c bytes) 2024-09-21T15:33:33.383 INFO:teuthology.orchestra.run.smithi107.stdout:2387 write 0xb73dc5 thru 0xb8228e (0xe4ca bytes) 2024-09-21T15:33:33.391 INFO:teuthology.orchestra.run.smithi107.stdout:2388 read 0x6ce6af8 thru 0x6cf1dcf (0xb2d8 bytes) 2024-09-21T15:33:33.391 INFO:teuthology.orchestra.run.smithi107.stdout:2389 read 0x5147e0c thru 0x5149f61 (0x2156 bytes) 2024-09-21T15:33:33.391 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.391 INFO:teuthology.orchestra.run.smithi107.stdout:2392 write 0x879667 thru 0x87e1c4 (0x4b5e bytes) 2024-09-21T15:33:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:2393 write 0x15dcff0 thru 0x15dda0a (0xa1b bytes) 2024-09-21T15:33:33.401 INFO:teuthology.orchestra.run.smithi107.stdout:2394 read 0x342b651 thru 0x342e052 (0x2a02 bytes) 2024-09-21T15:33:33.418 INFO:teuthology.orchestra.run.smithi107.stdout:2396 writesame 0xb4e951a thru 0xb4f12c3 data_size 0x5(0x7daa bytes) 2024-09-21T15:33:33.418 INFO:teuthology.orchestra.run.smithi107.stdout:2397 write 0x3cd50b1 thru 0x3ce0b7f (0xbacf bytes) 2024-09-21T15:33:33.425 INFO:teuthology.orchestra.run.smithi107.stdout:2398 write 0x501206a thru 0x501daba (0xba51 bytes) 2024-09-21T15:33:33.431 INFO:teuthology.orchestra.run.smithi107.stdout:2399 read 0x2df6e35 thru 0x2df8756 (0x1922 bytes) 2024-09-21T15:33:33.431 INFO:teuthology.orchestra.run.smithi107.stdout:2400 write 0x79a55c thru 0x7a011a (0x5bbf bytes) 2024-09-21T15:33:33.440 INFO:teuthology.orchestra.run.smithi107.stdout:2401 clone 19 order 19 su 8192 sc 2 2024-09-21T15:33:33.473 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:33.473 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:33.482 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:33.478+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:33.485 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:33.528 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:33.582 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:33.582+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:33.642 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:33.642 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:33.655 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:33.659 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:33.660 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:33.676 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:33.676 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:33.683 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:33.679+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:33.692 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:33.708 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:33.725 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:33.725 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:33.745 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:33.746 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:33.764 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:33.764 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:33.764 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T15:33:33.764 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T15:33:33.765 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:33:33.768 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:33.768 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:33.784 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:33.779+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:33.791 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:33.791 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:33.792 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:33.792 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:33.792 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:33.792 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:33.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:33.840 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone18 from 0xb4f12c4 (overlap 0xe1c181) to 0x2d26b3 2024-09-21T15:33:33.885 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:33.882+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:33.932 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T15:33:33.933 INFO:teuthology.orchestra.run.smithi107.stdout:2403 write 0xed9f13 thru 0xee163d (0x772b bytes) 2024-09-21T15:33:33.935 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T15:33:33.938 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:33.938 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:33.938 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:33.939 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:33.939 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:33.953 INFO:teuthology.orchestra.run.smithi107.stdout:2404 read 0x513a1e0 thru 0x5143b76 (0x9997 bytes) 2024-09-21T15:33:33.953 INFO:teuthology.orchestra.run.smithi107.stdout:2406 writesame 0x9db8185 thru 0x9db8c61 data_size 0x9(0xadd bytes) 2024-09-21T15:33:33.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:33.968 INFO:teuthology.orchestra.run.smithi107.stdout:2407 write 0xe8304ff thru 0xe83a077 (0x9b79 bytes) 2024-09-21T15:33:33.973 INFO:teuthology.orchestra.run.smithi107.stdout:2408 read 0xbb5addb thru 0xbb5f7c4 (0x49ea bytes) 2024-09-21T15:33:33.973 INFO:teuthology.orchestra.run.smithi107.stdout:2409 punch from 0x4a496 to 0x518ab, (0x7415 bytes) 2024-09-21T15:33:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:2411 write 0x33474aa thru 0x3357474 (0xffcb bytes) 2024-09-21T15:33:33.978 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:33.978 INFO:teuthology.orchestra.run.smithi107.stdout:2413 write 0x23755ca thru 0x237bfe5 (0x6a1c bytes) 2024-09-21T15:33:33.983 INFO:teuthology.orchestra.run.smithi107.stdout:2414 read 0xd1b63a4 thru 0xd1bfc69 (0x98c6 bytes) 2024-09-21T15:33:33.983 INFO:teuthology.orchestra.run.smithi107.stdout:2415 flatten 2024-09-21T15:33:33.986 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:33.983+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:34.087 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:34.082+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:34.102 INFO:teuthology.orchestra.run.smithi107.stdout:2417 punch from 0x98e53a9 to 0x98e6429, (0x1080 bytes) 2024-09-21T15:33:34.102 INFO:teuthology.orchestra.run.smithi107.stdout:2419 read 0x3d013a5 thru 0x3d09c5f (0x88bb bytes) 2024-09-21T15:33:34.102 INFO:teuthology.orchestra.run.smithi107.stdout:2423 writesame 0x6c59291 thru 0x6c683ae data_size 0x7(0xf11e bytes) 2024-09-21T15:33:34.108 INFO:teuthology.orchestra.run.smithi107.stdout:2424 writesame 0xb15cf5f thru 0xb160476 data_size 0x4(0x3518 bytes) 2024-09-21T15:33:34.112 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.113 INFO:teuthology.orchestra.run.smithi107.stdout:2426 punch from 0x7e03d4 to 0x7eb9fd, (0xb629 bytes) 2024-09-21T15:33:34.113 INFO:teuthology.orchestra.run.smithi107.stdout:2428 read 0xd566964 thru 0xd572381 (0xba1e bytes) 2024-09-21T15:33:34.113 INFO:teuthology.orchestra.run.smithi107.stdout:2429 write 0x86d80a4 thru 0x86e0871 (0x87ce bytes) 2024-09-21T15:33:34.133 INFO:teuthology.orchestra.run.smithi107.stdout:2430 trunc from 0xe83a078 to 0xce0997 2024-09-21T15:33:34.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T15:33:34.146 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.471671612999998, 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.147 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:34.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:34.152 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.355574259999997, 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.318994146000001, 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:34.157 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:34.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.300065451999998, 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:34.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:34.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:34.166 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.560316421000003, 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:34.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.425094027999997, 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:34.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:34.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.422065298, 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.175 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.179 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.123859965000001, 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:34.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:34.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.185 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.073274327999997, 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:34.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.189 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 21.711563862999999, 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:34.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16689 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.195 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.14999427900000001, 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099936150000000008, 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16689 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997252+0000", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997257+0000", 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8420000000000004e-06 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997272+0000", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5102e-05 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997284+0000", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1616e-05 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006897+0000", 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096132590000000007 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007002+0000", 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010490200000000001 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007247+0000", 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024475099999999997 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16690 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.199 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.14995352200000001, 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010305910999999999, 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16690 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997293+0000", 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997300+0000", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8170000000000001e-06 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997315+0000", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4793e-05 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997327+0000", 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1953000000000001e-05 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007284+0000", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099574629999999997 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007388+0000", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103568 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007600+0000", 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000212143 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.203 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16692 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.14986787200000001, 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01059863, 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16692 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.204 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997379+0000", 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997385+0000", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3820000000000001e-06 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997400+0000", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5162e-05 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997412+0000", 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1749999999999999e-05 2024-09-21T15:33:34.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007635+0000", 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010222663 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007732+0000", 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7085999999999998e-05 2024-09-21T15:33:34.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007978+0000", 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024659200000000001 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16694 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.14977637299999999, 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010846381, 2024-09-21T15:33:34.208 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16694 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997470+0000", 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997475+0000", 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997490+0000", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5285e-05 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997502+0000", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.199e-05 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008013+0000", 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010510733 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008114+0000", 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100599 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008318+0000", 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020389400000000001 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16696 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.212 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.14969279899999999, 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011432552, 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16696 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:34.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997554+0000", 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997559+0000", 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.994e-06 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997576+0000", 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6767000000000001e-05 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997587+0000", 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1810999999999999e-05 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008339+0000", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010751805 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008417+0000", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7960000000000006e-05 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008987+0000", 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00057021800000000005 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.216 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16697 3.1a 3:5bcd11c0:::rbd_data.1191bd191516.0000000000000008:14 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.149650002, 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010924658, 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16697 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997597+0000", 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997602+0000", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997618+0000", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6535999999999999e-05 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008249+0000", 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010630707 2024-09-21T15:33:34.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008319+0000", 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0352999999999995e-05 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008403+0000", 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3627000000000004e-05 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008522+0000", 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119447 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16790 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.13846475, 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098152529999999995, 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.221 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16790 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008782+0000", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008786+0000", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6569999999999999e-06 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008796+0000", 2024-09-21T15:33:34.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8199999999999992e-06 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008803+0000", 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8609999999999998e-06 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018427+0000", 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096234849999999993 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018499+0000", 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1666000000000001e-05 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018598+0000", 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.946e-05 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16793 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:34.225 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.13843703800000001, 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099596430000000007, 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16793 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008810+0000", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008817+0000", 2024-09-21T15:33:34.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4189999999999996e-06 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008827+0000", 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008835+0000", 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8099999999999998e-06 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018630+0000", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097946439999999999 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018683+0000", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3457e-05 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018770+0000", 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6965999999999994e-05 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.229 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16796 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.13840488300000001, 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010146664, 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16796 2024-09-21T15:33:34.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008842+0000", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008846+0000", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5820000000000002e-06 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008856+0000", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0183000000000001e-05 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008866+0000", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018820+0000", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099546500000000007 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018894+0000", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3609000000000006e-05 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018989+0000", 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5456000000000002e-05 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:17308 3.1a 3:5e234bb8:::rbd_data.1191c93bd12d.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.11362006299999999, 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012577608000000001, 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.234 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 17308 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:34.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033627+0000", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034352+0000", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072501 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034367+0000", 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5798e-05 2024-09-21T15:33:34.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034375+0000", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6320000000000008e-06 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034583+0000", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020768800000000001 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034802+0000", 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021902600000000001 2024-09-21T15:33:34.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.035066+0000", 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026464199999999999 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.036863+0000", 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001796834 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046146+0000", 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092828340000000002 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046176+0000", 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9972999999999999e-05 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046205+0000", 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9161000000000001e-05 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.240 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.240 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.240 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2432 trunc from 0xce0997 to 0xa601d5f 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2435 write 0xea7d49c thru 0xea87a7b (0xa5e0 bytes) 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2436 trunc from 0xea87a7c to 0x1a0fbd0 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2438 trunc from 0x1a0fbd0 to 0xbaf7c4e 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2439 trunc from 0xbaf7c4e to 0x88dd010 2024-09-21T15:33:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:2440 trunc from 0x88dd010 to 0x458032f 2024-09-21T15:33:34.243 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:34.243 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:34.238+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:2441 read 0x38c58c3 thru 0x38cc8ce (0x700c bytes) 2024-09-21T15:33:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:2442 read 0x3d8b0de thru 0x3d95a54 (0xa977 bytes) 2024-09-21T15:33:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:2443 trunc from 0x458032f to 0x30d81c1 2024-09-21T15:33:34.256 INFO:teuthology.orchestra.run.smithi107.stdout:2444 trunc from 0x30d81c1 to 0x4478e38 2024-09-21T15:33:34.264 INFO:teuthology.orchestra.run.smithi107.stdout:2447 read 0x38b9360 thru 0x38be325 (0x4fc6 bytes) 2024-09-21T15:33:34.264 INFO:teuthology.orchestra.run.smithi107.stdout:2450 read 0x13104bf thru 0x13191f0 (0x8d32 bytes) 2024-09-21T15:33:34.286 INFO:teuthology.orchestra.run.smithi107.stdout:2451 write 0xedf4683 thru 0xedf6aae (0x242c bytes) 2024-09-21T15:33:34.293 INFO:teuthology.orchestra.run.smithi107.stdout:2454 writesame 0xd4f75a5 thru 0xd506ab2 data_size 0x7(0xf50e bytes) 2024-09-21T15:33:34.309 INFO:teuthology.orchestra.run.smithi107.stdout:2456 write 0x615d89f thru 0x6164bae (0x7310 bytes) 2024-09-21T15:33:34.317 INFO:teuthology.orchestra.run.smithi107.stdout:2457 read 0xd2ccddb thru 0xd2d643d (0x9663 bytes) 2024-09-21T15:33:34.317 INFO:teuthology.orchestra.run.smithi107.stdout:2458 write 0x521090c thru 0x521d966 (0xd05b bytes) 2024-09-21T15:33:34.331 INFO:teuthology.orchestra.run.smithi107.stdout:2459 trunc from 0xedf6aaf to 0xbc57ebf 2024-09-21T15:33:34.342 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:34.339+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:34.348 INFO:teuthology.orchestra.run.smithi107.stdout:2460 write 0x7a7a8db thru 0x7a835ce (0x8cf4 bytes) 2024-09-21T15:33:34.355 INFO:teuthology.orchestra.run.smithi107.stdout:2461 read 0x80d1418 thru 0x80de0b0 (0xcc99 bytes) 2024-09-21T15:33:34.366 INFO:teuthology.orchestra.run.smithi107.stdout:2462 trunc from 0xbc57ebf to 0x6b06719 2024-09-21T15:33:34.381 INFO:teuthology.orchestra.run.smithi107.stdout:2463 punch from 0x162191a to 0x16300ac, (0xe792 bytes) 2024-09-21T15:33:34.394 INFO:teuthology.orchestra.run.smithi107.stdout:2464 write 0xbeee9b3 thru 0xbef60c1 (0x770f bytes) 2024-09-21T15:33:34.400 INFO:teuthology.orchestra.run.smithi107.stdout:2465 writesame 0x8b7e951 thru 0x8b7ee9c data_size 0x6(0x54c bytes) 2024-09-21T15:33:34.404 INFO:teuthology.orchestra.run.smithi107.stdout:2466 write 0x1c8bffa thru 0x1c8f719 (0x3720 bytes) 2024-09-21T15:33:34.410 INFO:teuthology.orchestra.run.smithi107.stdout:2469 read 0xa2d3f43 thru 0xa2d7955 (0x3a13 bytes) 2024-09-21T15:33:34.413 INFO:teuthology.orchestra.run.smithi107.stdout:2470 trunc from 0xbef60c2 to 0x9992aa3 2024-09-21T15:33:34.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T15:33:34.424 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:34.424 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:34.424 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:34.424 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:34.427 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:34.440 INFO:teuthology.orchestra.run.smithi107.stdout:2472 write 0xe3a6e2b thru 0xe3a7af9 (0xccf bytes) 2024-09-21T15:33:34.444 INFO:teuthology.orchestra.run.smithi107.stdout:2473 write 0xa2ceb0d thru 0xa2db7e3 (0xccd7 bytes) 2024-09-21T15:33:34.449 INFO:teuthology.orchestra.run.smithi107.stdout:2474 punch from 0x19ac820 to 0x19b833c, (0xbb1c bytes) 2024-09-21T15:33:34.450 INFO:teuthology.orchestra.run.smithi107.stdout:2475 write 0x25adaa4 thru 0x25b47cd (0x6d2a bytes) 2024-09-21T15:33:34.456 INFO:teuthology.orchestra.run.smithi107.stdout:2476 punch from 0x997afec to 0x997db37, (0x2b4b bytes) 2024-09-21T15:33:34.456 INFO:teuthology.orchestra.run.smithi107.stdout:2477 read 0xc37a855 thru 0xc384e75 (0xa621 bytes) 2024-09-21T15:33:34.456 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.456 INFO:teuthology.orchestra.run.smithi107.stdout:2479 write 0xca944e8 thru 0xcaa3745 (0xf25e bytes) 2024-09-21T15:33:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:2481 trunc from 0xe3a7afa to 0x76e5437 2024-09-21T15:33:34.479 INFO:teuthology.orchestra.run.smithi107.stdout:2483 punch from 0x6c50ad9 to 0x6c5aafb, (0xa022 bytes) 2024-09-21T15:33:34.479 INFO:teuthology.orchestra.run.smithi107.stdout:2484 writesame 0x33f6bd3 thru 0x33feb31 data_size 0x3(0x7f5f bytes) 2024-09-21T15:33:34.484 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:34.482+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:34.493 INFO:teuthology.orchestra.run.smithi107.stdout:2485 write 0x7ca6381 thru 0x7ca67c7 (0x447 bytes) 2024-09-21T15:33:34.497 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.497 INFO:teuthology.orchestra.run.smithi107.stdout:2489 read 0x1a6e1c8 thru 0x1a735c7 (0x5400 bytes) 2024-09-21T15:33:34.504 INFO:teuthology.orchestra.run.smithi107.stdout:2490 trunc from 0x7ca67c8 to 0x278371d 2024-09-21T15:33:34.517 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.517 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:34.517 INFO:teuthology.orchestra.run.smithi107.stdout:2494 read 0x17ab112 thru 0x17b1010 (0x5eff bytes) 2024-09-21T15:33:34.517 INFO:teuthology.orchestra.run.smithi107.stdout:2495 punch from 0x1e5242a to 0x1e609fa, (0xe5d0 bytes) 2024-09-21T15:33:34.517 INFO:teuthology.orchestra.run.smithi107.stdout:2496 clone 20 order 24 su 8388608 sc 6 2024-09-21T15:33:34.544 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:34.542+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:34.563 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:34.563 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:34.563 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:34.563 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:34.563 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:34.577 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:34.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T15:33:34.676 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:34.675+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:34.746 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:34.746 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:34.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:34.746 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 57.958650714000001, 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.747 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:34.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:34.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.952039507999999, 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:34.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:34.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:34.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.377211789, 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:34.761 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:34.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:34.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:34.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:872 3.5 3:a31b1cfe:::rbd_data.11911fd826fd.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.752939341999998, 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0076432899999999996, 2024-09-21T15:33:34.767 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 872 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:34.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993095+0000", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993106+0000", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1588e-05 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993118+0000", 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1914e-05 2024-09-21T15:33:34.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993127+0000", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7059999999999993e-06 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993260+0000", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013325199999999999 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993527+0000", 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000266464 2024-09-21T15:33:34.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.994037+0000", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051008300000000004 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.996466+0000", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024291550000000001 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998600+0000", 2024-09-21T15:33:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021341989999999998 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998759+0000", 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000158269 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000739+0000", 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0019806120000000001 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:989 3.c 3:35c0a0a7:::rbd_data.11911fd826fd.0000000000000037:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.708653769000001, 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007642383, 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 989 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037379+0000", 2024-09-21T15:33:34.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037392+0000", 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2522e-05 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037411+0000", 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9052e-05 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037424+0000", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2822e-05 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037498+0000", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3934000000000001e-05 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037639+0000", 2024-09-21T15:33:34.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014103499999999999 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037876+0000", 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236782 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.039324+0000", 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001448392 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044959+0000", 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056346479999999999 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044994+0000", 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5728000000000001e-05 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045024+0000", 2024-09-21T15:33:34.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9522e-05 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.075397795999997, 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:34.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:34.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.996581141999997, 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:34.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:34.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.966228129000001, 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:34.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:34.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:34.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.958356434000002, 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:34.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.800 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.947176986000002, 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:34.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:34.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.941944620000001, 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.806 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:34.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:34.810 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.923459338999997, 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:34.814 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.796305836999998, 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:34.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:34.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.796149515000003, 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.820 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:34.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:34.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:34.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.790467290000002, 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.826 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:34.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:34.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:34.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.831 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.789504471000001, 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:34.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.028659548, 2024-09-21T15:33:34.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:34.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:34.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.719634914, 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:34.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:34.847 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.440569936999999, 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:34.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:34.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:34.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:34.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.10973044, 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:34.863 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:34.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:34.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:34.870 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:34.871 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:34.742+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:34.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:34.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-21T15:33:34.917 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:34.914+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:34.931 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T15:33:34.940 INFO:teuthology.orchestra.run.smithi107.stdout:2497 punch from 0x192d124 to 0x193a189, (0xd065 bytes) 2024-09-21T15:33:34.946 INFO:teuthology.orchestra.run.smithi107.stdout:2498 read 0x151c547 thru 0x1523567 (0x7021 bytes) 2024-09-21T15:33:34.968 INFO:teuthology.orchestra.run.smithi107.stdout:2499 write 0xa8773cb thru 0xa87e704 (0x733a bytes) 2024-09-21T15:33:34.973 INFO:teuthology.orchestra.run.smithi107.stdout:2500 write 0x40c9670 thru 0x40d0fdf (0x7970 bytes) 2024-09-21T15:33:34.974 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:34.970+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:34.978 INFO:teuthology.orchestra.run.smithi107.stdout:2501 write 0x54d7013 thru 0x54d7175 (0x163 bytes) 2024-09-21T15:33:34.983 INFO:teuthology.orchestra.run.smithi107.stdout:2502 punch from 0x320623d to 0x320d844, (0x7607 bytes) 2024-09-21T15:33:34.996 INFO:teuthology.orchestra.run.smithi107.stdout:2503 write 0xcbe237c thru 0xcbe854d (0x61d2 bytes) 2024-09-21T15:33:35.001 INFO:teuthology.orchestra.run.smithi107.stdout:2504 write 0x5cf9171 thru 0x5cfa0b2 (0xf42 bytes) 2024-09-21T15:33:35.005 INFO:teuthology.orchestra.run.smithi107.stdout:2505 read 0x49f7d70 thru 0x4a01469 (0x96fa bytes) 2024-09-21T15:33:35.006 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.006 INFO:teuthology.orchestra.run.smithi107.stdout:2508 writesame 0x233de2f thru 0x234810c data_size 0x3(0xa2de bytes) 2024-09-21T15:33:35.024 INFO:teuthology.orchestra.run.smithi107.stdout:2509 trunc from 0xcbe854e to 0x824b041 2024-09-21T15:33:35.055 INFO:teuthology.orchestra.run.smithi107.stdout:2510 write 0xaa67ca3 thru 0xaa71f84 (0xa2e2 bytes) 2024-09-21T15:33:35.056 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.056 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:35.056 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:35.056 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:35.057 INFO:teuthology.orchestra.run.smithi107.stdout:2512 write 0x240b2ef thru 0x2414af4 (0x9806 bytes) 2024-09-21T15:33:35.063 INFO:teuthology.orchestra.run.smithi107.stdout:2513 trunc from 0xaa71f85 to 0xd970c29 2024-09-21T15:33:35.071 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:35.072 INFO:teuthology.orchestra.run.smithi107.stdout:2514 write 0x12d00b8 thru 0x12dcbf2 (0xcb3b bytes) 2024-09-21T15:33:35.073 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.074 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:35.071+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:35.081 INFO:teuthology.orchestra.run.smithi107.stdout:2516 write 0xe790bd0 thru 0xe794cf9 (0x412a bytes) 2024-09-21T15:33:35.085 INFO:teuthology.orchestra.run.smithi107.stdout:2518 punch from 0x47f4f27 to 0x47ffedd, (0xafb6 bytes) 2024-09-21T15:33:35.085 INFO:teuthology.orchestra.run.smithi107.stdout:2519 write 0x3b1e322 thru 0x3b225a2 (0x4281 bytes) 2024-09-21T15:33:35.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-21T15:33:35.105 INFO:teuthology.orchestra.run.smithi107.stdout:2520 write 0xa65475d thru 0xa657fb3 (0x3857 bytes) 2024-09-21T15:33:35.108 INFO:teuthology.orchestra.run.smithi107.stdout:2521 writesame 0x1c4b00d thru 0x1c4d8a0 data_size 0x35(0x2894 bytes) 2024-09-21T15:33:35.113 INFO:teuthology.orchestra.run.smithi107.stdout:2523 punch from 0x3b2c61d to 0x3b31dc3, (0x57a6 bytes) 2024-09-21T15:33:35.113 INFO:teuthology.orchestra.run.smithi107.stdout:2524 read 0xb8fbec thru 0xb9aacf (0xaee4 bytes) 2024-09-21T15:33:35.113 INFO:teuthology.orchestra.run.smithi107.stdout:2525 write 0x18f2f69 thru 0x1901363 (0xe3fb bytes) 2024-09-21T15:33:35.124 INFO:teuthology.orchestra.run.smithi107.stdout:2526 trunc from 0xe794cfa to 0x8593b94 2024-09-21T15:33:35.148 INFO:teuthology.orchestra.run.smithi107.stdout:2527 writesame 0xad7dc45 thru 0xad8c135 data_size 0x1(0xe4f1 bytes) 2024-09-21T15:33:35.160 INFO:teuthology.orchestra.run.smithi107.stdout:2529 write 0xcbda8b5 thru 0xcbdbf57 (0x16a3 bytes) 2024-09-21T15:33:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:2530 write 0xe53c04 thru 0xe62fe0 (0xf3dd bytes) 2024-09-21T15:33:35.165 INFO:teuthology.orchestra.run.smithi107.stdout:2531 read 0xbbce673 thru 0xbbdca5b (0xe3e9 bytes) 2024-09-21T15:33:35.174 INFO:teuthology.orchestra.run.smithi107.stdout:2532 write 0xebc1393 thru 0xebc498f (0x35fd bytes) 2024-09-21T15:33:35.175 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:35.171+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:2533 write 0xb8c9990 thru 0xb8d70e3 (0xd754 bytes) 2024-09-21T15:33:35.181 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.181 INFO:teuthology.orchestra.run.smithi107.stdout:2537 write 0x63dbccf thru 0x63ddf32 (0x2264 bytes) 2024-09-21T15:33:35.184 INFO:teuthology.orchestra.run.smithi107.stdout:2538 write 0x631c05c thru 0x6320f20 (0x4ec5 bytes) 2024-09-21T15:33:35.186 INFO:teuthology.orchestra.run.smithi107.stdout:2539 punch from 0x3ae97c4 to 0x3af4883, (0xb0bf bytes) 2024-09-21T15:33:35.186 INFO:teuthology.orchestra.run.smithi107.stdout:2541 punch from 0x1af7792 to 0x1b06ea4, (0xf712 bytes) 2024-09-21T15:33:35.186 INFO:teuthology.orchestra.run.smithi107.stdout:2543 write 0xa6ba5f7 thru 0xa6c22ed (0x7cf7 bytes) 2024-09-21T15:33:35.187 INFO:teuthology.orchestra.run.smithi107.stdout:2544 read 0xdea8259 thru 0xdeaca11 (0x47b9 bytes) 2024-09-21T15:33:35.188 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.208 INFO:teuthology.orchestra.run.smithi107.stdout:2547 trunc from 0xebc4990 to 0x69d081 2024-09-21T15:33:35.253 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.253 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:35.253 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:35.253 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:35.253 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:35.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:35.275 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:35.271+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:35.278 INFO:teuthology.orchestra.run.smithi107.stdout:2548 writesame 0xe827a21 thru 0xe82b903 data_size 0x1(0x3ee3 bytes) 2024-09-21T15:33:35.278 INFO:teuthology.orchestra.run.smithi107.stdout:2549 read 0x2968a21 thru 0x29760f4 (0xd6d4 bytes) 2024-09-21T15:33:35.279 INFO:teuthology.orchestra.run.smithi107.stdout:2550 writesame 0x11a4232 thru 0x11ae8cc data_size 0x15(0xa69b bytes) 2024-09-21T15:33:35.283 INFO:teuthology.orchestra.run.smithi107.stdout:2551 write 0xe450168 thru 0xe45acf8 (0xab91 bytes) 2024-09-21T15:33:35.297 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T15:33:35.300 INFO:teuthology.orchestra.run.smithi107.stdout:2552 trunc from 0xe82b904 to 0x6810b72 2024-09-21T15:33:35.326 INFO:teuthology.orchestra.run.smithi107.stdout:2554 write 0xb8ba591 thru 0xb8c7bf4 (0xd664 bytes) 2024-09-21T15:33:35.330 INFO:teuthology.orchestra.run.smithi107.stdout:2555 read 0x33e1212 thru 0x33f0ce8 (0xfad7 bytes) 2024-09-21T15:33:35.331 INFO:teuthology.orchestra.run.smithi107.stdout:2558 read 0xc17089 thru 0xc23ce9 (0xcc61 bytes) 2024-09-21T15:33:35.331 INFO:teuthology.orchestra.run.smithi107.stdout:2559 punch from 0x80df62b to 0x80e717d, (0x7b52 bytes) 2024-09-21T15:33:35.331 INFO:teuthology.orchestra.run.smithi107.stdout:2560 punch from 0x4ce33bc to 0x4cec0d1, (0x8d15 bytes) 2024-09-21T15:33:35.346 INFO:teuthology.orchestra.run.smithi107.stdout:2561 trunc from 0xb8c7bf5 to 0xc99827 2024-09-21T15:33:35.364 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.373 INFO:teuthology.orchestra.run.smithi107.stdout:2565 write 0x1eb4752 thru 0x1ebc0dc (0x798b bytes) 2024-09-21T15:33:35.376 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:35.375+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:35.377 INFO:teuthology.orchestra.run.smithi107.stdout:2567 read 0x8fd750 thru 0x903c79 (0x652a bytes) 2024-09-21T15:33:35.377 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.377 INFO:teuthology.orchestra.run.smithi107.stdout:2571 punch from 0x15f4261 to 0x15fcc83, (0x8a22 bytes) 2024-09-21T15:33:35.377 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.377 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.378 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.378 INFO:teuthology.orchestra.run.smithi107.stdout:2575 punch from 0x19a66e1 to 0x19b13c0, (0xacdf bytes) 2024-09-21T15:33:35.378 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.378 INFO:teuthology.orchestra.run.smithi107.stdout:2577 read 0xaf39fc thru 0xaf4d3a (0x133f bytes) 2024-09-21T15:33:35.378 INFO:teuthology.orchestra.run.smithi107.stdout:2578 read 0x13df546 thru 0x13e7432 (0x7eed bytes) 2024-09-21T15:33:35.395 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.930403446, 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:35.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:35.400 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.70012607, 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:35.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:35.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:35.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:35.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.398755279, 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.406 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:35.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:35.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:35.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.723241459, 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.412 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:35.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:35.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:35.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.394385097, 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3943686049999999, 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:35.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.394352233, 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.430 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.394335568, 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:35.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3943187459999999, 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:35.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.393990828, 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:35.439 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:35.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3939601859999999, 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.444 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:35.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3939420419999999, 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.393911586, 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:35.452 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:35.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3938932980000001, 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:35.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3938760100000001, 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:35.461 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:35.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:35.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.465 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.393859344, 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:35.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:35.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3938349969999999, 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:35.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:35.474 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3938188550000001, 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:35.478 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3938023099999999, 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:35.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:35.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.3937747659999999, 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:35.483 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.487 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:35.490 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:35.490 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:35.486+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:35.491 INFO:teuthology.orchestra.run.smithi107.stdout:2580 writesame 0x8d9e84f thru 0x8daa3a3 data_size 0x77(0xbb55 bytes) 2024-09-21T15:33:35.491 INFO:teuthology.orchestra.run.smithi107.stdout:2581 read 0x85720e7 thru 0x8577006 (0x4f20 bytes) 2024-09-21T15:33:35.491 INFO:teuthology.orchestra.run.smithi107.stdout:2582 write 0x2fa9301 thru 0x2fabd10 (0x2a10 bytes) 2024-09-21T15:33:35.491 INFO:teuthology.orchestra.run.smithi107.stdout:2583 writesame 0x10ce68e thru 0x10d430f data_size 0x3(0x5c82 bytes) 2024-09-21T15:33:35.491 INFO:teuthology.orchestra.run.smithi107.stdout:2584 write 0x1299278 thru 0x129d838 (0x45c1 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2585 punch from 0x3facb0 to 0x404d5a, (0xa0aa bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2587 write 0xd9d3d02 thru 0xd9e07c7 (0xcac6 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2588 read 0x98c61a2 thru 0x98ceee4 (0x8d43 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2589 writesame 0x29c358 thru 0x2a174d data_size 0xb(0x53f6 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2590 write 0x4f3055 thru 0x4ff227 (0xc1d3 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2591 trunc from 0xd9e07c8 to 0xdb6309e 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2592 write 0xa775a3b thru 0xa77fd17 (0xa2dd bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2593 read 0x9f18c45 thru 0x9f23b0b (0xaec7 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2594 punch from 0x5093ea1 to 0x50a32b7, (0xf416 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2595 punch from 0x3ab4aaa to 0x3ac3dcf, (0xf325 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2598 read 0x1f85bdc thru 0x1f8e560 (0x8985 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2599 write 0x14b7e20 thru 0x14ba480 (0x2661 bytes) 2024-09-21T15:33:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:2600 trunc from 0xdb6309e to 0x6152bc2 2024-09-21T15:33:35.590 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:35.587+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:35.661 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.661 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:35.661 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:35.661 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:35.675 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:35.687 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.688 INFO:teuthology.orchestra.run.smithi107.stdout:2602 trunc from 0x6152bc2 to 0x5ce9d44 2024-09-21T15:33:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:2603 trunc from 0x5ce9d44 to 0x70c911b 2024-09-21T15:33:35.713 INFO:teuthology.orchestra.run.smithi107.stdout:2604 read 0x3b993d8 thru 0x3ba3da3 (0xa9cc bytes) 2024-09-21T15:33:35.713 INFO:teuthology.orchestra.run.smithi107.stdout:2605 read 0x35817d2 thru 0x3587fc1 (0x67f0 bytes) 2024-09-21T15:33:35.714 INFO:teuthology.orchestra.run.smithi107.stdout:2606 write 0x285131d thru 0x285794a (0x662e bytes) 2024-09-21T15:33:35.724 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:35.722+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:35.730 INFO:teuthology.orchestra.run.smithi107.stdout:2607 trunc from 0x70c911b to 0xc5002f0 2024-09-21T15:33:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:2608 writesame 0x4d9847c thru 0x4d9b932 data_size 0x1(0x34b7 bytes) 2024-09-21T15:33:35.757 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.757 INFO:teuthology.orchestra.run.smithi107.stdout:2610 read 0x44c1fb1 thru 0x44c3526 (0x1576 bytes) 2024-09-21T15:33:35.777 INFO:teuthology.orchestra.run.smithi107.stdout:2611 write 0xe8df217 thru 0xe8e5761 (0x654b bytes) 2024-09-21T15:33:35.786 INFO:teuthology.orchestra.run.smithi107.stdout:2612 punch from 0x66c0499 to 0x66c6499, (0x6000 bytes) 2024-09-21T15:33:35.786 INFO:teuthology.orchestra.run.smithi107.stdout:2613 read 0xc9c5227 thru 0xc9d010c (0xaee6 bytes) 2024-09-21T15:33:35.786 INFO:teuthology.orchestra.run.smithi107.stdout:2614 read 0xaff33eb thru 0xaff892b (0x5541 bytes) 2024-09-21T15:33:35.786 INFO:teuthology.orchestra.run.smithi107.stdout:2615 write 0xe59bcd4 thru 0xe5a6799 (0xaac6 bytes) 2024-09-21T15:33:35.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:35.793 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.793 INFO:teuthology.orchestra.run.smithi107.stdout:2618 read 0x1ad5a26 thru 0x1add729 (0x7d04 bytes) 2024-09-21T15:33:35.793 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:35.791+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:35.794 INFO:teuthology.orchestra.run.smithi107.stdout:2619 write 0x2931103 thru 0x293ccc2 (0xbbc0 bytes) 2024-09-21T15:33:35.798 INFO:teuthology.orchestra.run.smithi107.stdout:2621 punch from 0x632605e to 0x6332eed, (0xce8f bytes) 2024-09-21T15:33:35.798 INFO:teuthology.orchestra.run.smithi107.stdout:2623 write 0xc860e16 thru 0xc8681cc (0x73b7 bytes) 2024-09-21T15:33:35.799 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:35.799 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:35.804 INFO:teuthology.orchestra.run.smithi107.stdout:2624 read 0xd6b7612 thru 0xd6bc23c (0x4c2b bytes) 2024-09-21T15:33:35.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:35.811 INFO:teuthology.orchestra.run.smithi107.stdout:2627 trunc from 0xe8e5762 to 0xb183cbd 2024-09-21T15:33:35.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:35.830 INFO:teuthology.orchestra.run.smithi107.stdout:2628 punch from 0x69c1cb7 to 0x69c2edd, (0x1226 bytes) 2024-09-21T15:33:35.830 INFO:teuthology.orchestra.run.smithi107.stdout:2629 read 0x2b76d30 thru 0x2b785cf (0x18a0 bytes) 2024-09-21T15:33:35.831 INFO:teuthology.orchestra.run.smithi107.stdout:2630 punch from 0x2a8c600 to 0x2a8feb0, (0x38b0 bytes) 2024-09-21T15:33:35.831 INFO:teuthology.orchestra.run.smithi107.stdout:2631 read 0xf0cd9e thru 0xf0f5c7 (0x282a bytes) 2024-09-21T15:33:35.831 INFO:teuthology.orchestra.run.smithi107.stdout:2632 read 0x654ac5f thru 0x655156a (0x690c bytes) 2024-09-21T15:33:35.831 INFO:teuthology.orchestra.run.smithi107.stdout:2633 read 0xaad7142 thru 0xaae2f3b (0xbdfa bytes) 2024-09-21T15:33:35.831 INFO:teuthology.orchestra.run.smithi107.stdout:2634 read 0x7c4a51f thru 0x7c586ed (0xe1cf bytes) 2024-09-21T15:33:35.847 INFO:teuthology.orchestra.run.smithi107.stdout:2635 trunc from 0xb183cbd to 0xbd05c7 2024-09-21T15:33:35.868 INFO:teuthology.orchestra.run.smithi107.stdout:2636 punch from 0x97493b to 0x97ef6f, (0xa634 bytes) 2024-09-21T15:33:35.868 INFO:teuthology.orchestra.run.smithi107.stdout:2637 punch from 0x120134 to 0x129bf1, (0x9abd bytes) 2024-09-21T15:33:35.878 INFO:teuthology.orchestra.run.smithi107.stdout:2640 write 0x2bbe982 thru 0x2bcc328 (0xd9a7 bytes) 2024-09-21T15:33:35.893 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:35.891+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:35.898 INFO:teuthology.orchestra.run.smithi107.stdout:2642 write 0x8f7660a thru 0x8f7c276 (0x5c6d bytes) 2024-09-21T15:33:35.901 INFO:teuthology.orchestra.run.smithi107.stdout:2643 read 0x3a0e041 thru 0x3a13e7b (0x5e3b bytes) 2024-09-21T15:33:35.901 INFO:teuthology.orchestra.run.smithi107.stdout:2644 writesame 0x55dad3b thru 0x55e4211 data_size 0xd(0x94d7 bytes) 2024-09-21T15:33:35.906 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.906 INFO:teuthology.orchestra.run.smithi107.stdout:2647 punch from 0x7fcb78c to 0x7fd6344, (0xabb8 bytes) 2024-09-21T15:33:35.906 INFO:teuthology.orchestra.run.smithi107.stdout:2648 read 0x2ceacad thru 0x2cf127a (0x65ce bytes) 2024-09-21T15:33:35.917 INFO:teuthology.orchestra.run.smithi107.stdout:2649 writesame 0xb041df7 thru 0xb04bebd data_size 0x1(0xa0c7 bytes) 2024-09-21T15:33:35.917 INFO:teuthology.orchestra.run.smithi107.stdout:2650 read 0x250606a thru 0x2515f6c (0xff03 bytes) 2024-09-21T15:33:35.917 INFO:teuthology.orchestra.run.smithi107.stdout:2654 read 0x65b4359 thru 0x65bb710 (0x73b8 bytes) 2024-09-21T15:33:35.927 INFO:teuthology.orchestra.run.smithi107.stdout:2655 trunc from 0xb04bebe to 0x466555c 2024-09-21T15:33:35.942 INFO:teuthology.orchestra.run.smithi107.stdout:2658 read 0x301f97f thru 0x302a6bd (0xad3f bytes) 2024-09-21T15:33:35.942 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.942 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:35.946 INFO:teuthology.orchestra.run.smithi107.stdout:2662 trunc from 0x466555c to 0x7535e6f 2024-09-21T15:33:35.967 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:35.967 INFO:teuthology.orchestra.run.smithi107.stdout:2665 write 0xaed481e thru 0xaedb1df (0x69c2 bytes) 2024-09-21T15:33:35.970 INFO:teuthology.orchestra.run.smithi107.stdout:2666 writesame 0x27f2bdc thru 0x27faabf data_size 0x6(0x7ee4 bytes) 2024-09-21T15:33:35.975 INFO:teuthology.orchestra.run.smithi107.stdout:2667 punch from 0x30210c7 to 0x302a9d1, (0x990a bytes) 2024-09-21T15:33:35.981 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 64.917089462000007, 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:33:35.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:33:35.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:35.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.987 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1190 3.11 3:8bc4bd9b:::rbd_data.1191bb6b225c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.468616533999999, 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022028537000000001, 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1190 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.988 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512924+0000", 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:35.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512937+0000", 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3223e-05 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512966+0000", 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9215999999999999e-05 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512980+0000", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4273e-05 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513106+0000", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012556000000000001 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513349+0000", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000243341 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:35.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.517115+0000", 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003766138 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534677+0000", 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017561961000000001 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534756+0000", 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8382999999999998e-05 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534846+0000", 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0075999999999996e-05 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534954+0000", 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108446 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1220 3.7 e27/21 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head v 27'13, mlcod=27'11)", 2024-09-21T15:33:35.993 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.381383583999998, 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035610309, 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:35.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600156+0000", 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615711+0000", 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015554596 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615756+0000", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5343000000000003e-05 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615775+0000", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8564e-05 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615881+0000", 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010644199999999999 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615921+0000", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9133e-05 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635654+0000", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019733581 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635769+0000", 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0001149 2024-09-21T15:33:35.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1278 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.196989291000001, 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.060972842999999999, 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:35.998 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1278 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:35.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784551+0000", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801400+0000", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01684892 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801431+0000", 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1581999999999997e-05 2024-09-21T15:33:36.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801451+0000", 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.948e-05 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801589+0000", 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013792500000000001 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801994+0000", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040495700000000001 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.802913+0000", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00091958600000000001 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.832473+0000", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029559932000000001 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:36.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845354+0000", 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012880796999999999 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845448+0000", 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3746000000000007e-05 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845526+0000", 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7972999999999994e-05 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3973 3.e 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11281964~20966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.413396294000002, 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.059230786000000001, 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3973 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568144+0000", 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583870+0000", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015726602999999999 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583922+0000", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1239000000000002e-05 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583942+0000", 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0213000000000001e-05 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584121+0000", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000179469 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584383+0000", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026151100000000002 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.585420+0000", 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0010370189999999999 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.625790+0000", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040369751000000002 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627101+0000", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001311127 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627255+0000", 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015414899999999999 2024-09-21T15:33:36.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627377+0000", 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122009 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4208 3.11 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.272061612000002, 2024-09-21T15:33:36.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054071906000000003, 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4208 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709478+0000", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721564+0000", 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012085933 2024-09-21T15:33:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721605+0000", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0862999999999998e-05 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721625+0000", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9953e-05 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721772+0000", 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014629199999999999 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722036+0000", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000264415 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.723537+0000", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015015549999999999 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762221+0000", 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038683842000000003 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763323+0000", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011017150000000001 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763453+0000", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130199 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763553+0000", 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9382000000000005e-05 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14128 3.11 3:8a0b4402:::rbd_data.1191bd191516.000000000000003e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 11.365926706, 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045981519999999998, 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14128 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615614+0000", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615629+0000", 2024-09-21T15:33:36.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4788e-05 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615645+0000", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6401000000000001e-05 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615654+0000", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6160000000000002e-06 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615738+0000", 2024-09-21T15:33:36.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4012999999999997e-05 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615954+0000", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021574100000000001 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.625676+0000", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097218289999999995 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661339+0000", 2024-09-21T15:33:36.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035663551000000002 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661449+0000", 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109675 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661536+0000", 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7069000000000003e-05 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661597+0000", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1199999999999997e-05 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16747 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 319488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991660472, 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016176724999999999, 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.020 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16747 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989881+0000", 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989884+0000", 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.356e-06 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989890+0000", 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6640000000000003e-06 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989894+0000", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3749999999999996e-06 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.005838+0000", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015944100999999999 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.005947+0000", 2024-09-21T15:33:36.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108707 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006059+0000", 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011188000000000001 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16749 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9916442029999999, 2024-09-21T15:33:36.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016557470000000001, 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16749 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989900+0000", 2024-09-21T15:33:36.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1129999999999999e-06 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989906+0000", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.609e-06 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989910+0000", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4050000000000002e-06 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006159+0000", 2024-09-21T15:33:36.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016249257999999999 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006314+0000", 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154869 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006456+0000", 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014155299999999999 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16751 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9916282889999999, 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01678932, 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16751 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989916+0000", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.114e-06 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989922+0000", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8950000000000003e-06 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989926+0000", 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:33:36.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006503+0000", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016576911 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006602+0000", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9429e-05 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006703+0000", 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010092 2024-09-21T15:33:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16753 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9916120340000001, 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017122808999999999, 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.033 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16753 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989932+0000", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3029999999999998e-06 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989938+0000", 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0009999999999998e-06 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989943+0000", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.578e-06 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006767+0000", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016823683999999998 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006915+0000", 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014854899999999999 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007053+0000", 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013799600000000001 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16755 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9915954390000001, 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017350775999999998, 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.037 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16755 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989949+0000", 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.216e-06 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989955+0000", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989959+0000", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5709999999999999e-06 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007125+0000", 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017165369999999999 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007218+0000", 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3270999999999995e-05 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007298+0000", 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9561999999999995e-05 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16758 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.041 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9915577770000001, 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019563328000000001, 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16758 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:36.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989984+0000", 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989986+0000", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0810000000000001e-06 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989992+0000", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.789e-06 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989996+0000", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.374e-06 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007352+0000", 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017355085999999999 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007494+0000", 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000142936 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009548+0000", 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020534360000000001 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16760 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991525226, 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017899899, 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16760 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.046 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990019+0000", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1720000000000001e-06 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990025+0000", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7989999999999999e-06 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990029+0000", 2024-09-21T15:33:36.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4120000000000003e-06 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007660+0000", 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017630959000000002 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007804+0000", 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014346000000000001 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007917+0000", 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011346899999999999 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16762 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9914924940000001, 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018138254999999999, 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.050 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16762 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990049+0000", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990052+0000", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.881e-06 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990058+0000", 2024-09-21T15:33:36.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9730000000000002e-06 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990063+0000", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6990000000000002e-06 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007955+0000", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017891503999999999 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008077+0000", 2024-09-21T15:33:36.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122077 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008188+0000", 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111501 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16763 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:36.054 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991475069, 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018389483000000002, 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16763 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990069+0000", 2024-09-21T15:33:36.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0959999999999999e-06 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990075+0000", 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1519999999999997e-06 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990080+0000", 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5159999999999996e-06 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008226+0000", 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01814613 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008342+0000", 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011653499999999999 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008457+0000", 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114404 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16765 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991444711, 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018629436999999999, 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16765 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990097+0000", 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990099+0000", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1849999999999998e-06 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990105+0000", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990110+0000", 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6040000000000003e-06 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008495+0000", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018384482000000001 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008618+0000", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012336700000000001 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008727+0000", 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109144 2024-09-21T15:33:36.062 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16767 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.9913951139999999, 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018847929999999999, 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.063 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16767 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:36.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990149+0000", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.323e-06 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990155+0000", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8479999999999997e-06 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990160+0000", 2024-09-21T15:33:36.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6419999999999998e-06 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008763+0000", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018603831000000001 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008884+0000", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120865 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008995+0000", 2024-09-21T15:33:36.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110802 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16769 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991362412, 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019085418, 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.067 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16769 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990186+0000", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4989999999999999e-06 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990193+0000", 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.849e-06 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990198+0000", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8760000000000004e-06 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009032+0000", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018834178 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009154+0000", 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012181299999999999 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009265+0000", 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011154000000000001 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16770 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.991340109, 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019331745000000001, 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:36.071 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16770 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:36.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990204+0000", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.198e-06 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990209+0000", 2024-09-21T15:33:36.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990212+0000", 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5629999999999999e-06 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009301+0000", 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019089565999999999 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:36.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009420+0000", 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119072 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009534+0000", 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113731 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:36.075 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:36.076 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:36.076 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:36.078 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:36.074+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:2668 write 0xe1ab936 thru 0xe1abba2 (0x26d bytes) 2024-09-21T15:33:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:2669 read 0x10287c4 thru 0x102b026 (0x2863 bytes) 2024-09-21T15:33:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:2670 write 0x5e9c36c thru 0x5ea99c5 (0xd65a bytes) 2024-09-21T15:33:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:2671 read 0x8e0e3c2 thru 0x8e1d389 (0xefc8 bytes) 2024-09-21T15:33:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:2673 writesame 0x16f9afc thru 0x1700164 data_size 0x9(0x6669 bytes) 2024-09-21T15:33:36.078 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:36.079 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:36.079 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:36.080 INFO:teuthology.orchestra.run.smithi107.stdout:2675 read 0x147105d thru 0x1476679 (0x561d bytes) 2024-09-21T15:33:36.080 INFO:teuthology.orchestra.run.smithi107.stdout:2676 read 0xa58235d thru 0xa58ae18 (0x8abc bytes) 2024-09-21T15:33:36.080 INFO:teuthology.orchestra.run.smithi107.stdout:2677 write 0xd829450 thru 0xd82de44 (0x49f5 bytes) 2024-09-21T15:33:36.080 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:36.080 INFO:teuthology.orchestra.run.smithi107.stdout:2679 punch from 0xaba526c to 0xabae08c, (0x8e20 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2680 write 0x7d695e thru 0x7d9ade (0x3181 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2681 read 0x4aeab63 thru 0x4af2c36 (0x80d4 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2682 read 0x585b6cd thru 0x58658c5 (0xa1f9 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2683 read 0xd0993cc thru 0xd09db78 (0x47ad bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2684 write 0x3610671 thru 0x3610dce (0x75e bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2685 read 0xa042f2b thru 0xa0510c0 (0xe196 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:2686 writesame 0xde7fd5c thru 0xde8f7e0 data_size 0x1(0xfa85 bytes) 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:36.081 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:36.082 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:36.082 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:36.082 INFO:teuthology.orchestra.run.smithi107.stdout:2687 read 0x7922f92 thru 0x792f601 (0xc670 bytes) 2024-09-21T15:33:36.082 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:36.083 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:36.083 INFO:teuthology.orchestra.run.smithi107.stdout:2688 trunc from 0xe1abba3 to 0xb9188e 2024-09-21T15:33:36.083 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:36.084 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:36.097 INFO:teuthology.orchestra.run.smithi107.stdout:2689 read 0x26afa1 thru 0x2708fd (0x595d bytes) 2024-09-21T15:33:36.098 INFO:teuthology.orchestra.run.smithi107.stdout:2690 punch from 0x9d0a1a to 0x9d3aa6, (0x308c bytes) 2024-09-21T15:33:36.098 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:36.098 INFO:teuthology.orchestra.run.smithi107.stdout:2694 read 0x288030 thru 0x28fbf6 (0x7bc7 bytes) 2024-09-21T15:33:36.098 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:36.098 INFO:teuthology.orchestra.run.smithi107.stdout:2697 read 0x29e1be thru 0x2a9a58 (0xb89b bytes) 2024-09-21T15:33:36.102 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:36.103 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:36.134 INFO:teuthology.orchestra.run.smithi107.stdout:2698 write 0xe249e3d thru 0xe24c77c (0x2940 bytes) 2024-09-21T15:33:36.139 INFO:teuthology.orchestra.run.smithi107.stdout:2700 read 0x4dfb45d thru 0x4e06be3 (0xb787 bytes) 2024-09-21T15:33:36.139 INFO:teuthology.orchestra.run.smithi107.stdout:2702 write 0x548a96d thru 0x5494287 (0x991b bytes) 2024-09-21T15:33:36.144 INFO:teuthology.orchestra.run.smithi107.stdout:2703 writesame 0x604495b thru 0x6046710 data_size 0x1(0x1db6 bytes) 2024-09-21T15:33:36.157 INFO:teuthology.orchestra.run.smithi107.stdout:2704 read 0x93f2709 thru 0x93f528f (0x2b87 bytes) 2024-09-21T15:33:36.158 INFO:teuthology.orchestra.run.smithi107.stdout:2705 read 0xbdd638f thru 0xbdd978e (0x3400 bytes) 2024-09-21T15:33:36.158 INFO:teuthology.orchestra.run.smithi107.stdout:2706 read 0x32a906d thru 0x32b0b9b (0x7b2f bytes) 2024-09-21T15:33:36.159 INFO:teuthology.orchestra.run.smithi107.stdout:2708 writesame 0x1f6e7b thru 0x1fa9b4 data_size 0xe(0x3b3a bytes) 2024-09-21T15:33:36.161 INFO:teuthology.orchestra.run.smithi107.stdout:2709 punch from 0xd391c8e to 0xd393f6e, (0x22e0 bytes) 2024-09-21T15:33:36.161 INFO:teuthology.orchestra.run.smithi107.stdout:2712 write 0xc46d646 thru 0xc46fe75 (0x2830 bytes) 2024-09-21T15:33:36.168 INFO:teuthology.orchestra.run.smithi107.stdout:2715 write 0x8351502 thru 0x8353319 (0x1e18 bytes) 2024-09-21T15:33:36.175 INFO:teuthology.orchestra.run.smithi107.stdout:2716 read 0x253ceec thru 0x254ccbf (0xfdd4 bytes) 2024-09-21T15:33:36.179 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:36.174+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:36.179 INFO:teuthology.orchestra.run.smithi107.stdout:2717 punch from 0x385c6da to 0x3868873, (0xc199 bytes) 2024-09-21T15:33:36.180 INFO:teuthology.orchestra.run.smithi107.stdout:2719 read 0x6bb520b thru 0x6bc48e9 (0xf6df bytes) 2024-09-21T15:33:36.180 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:36.180 INFO:teuthology.orchestra.run.smithi107.stdout:2721 read 0x781f5a9 thru 0x7824d20 (0x5778 bytes) 2024-09-21T15:33:36.180 INFO:teuthology.orchestra.run.smithi107.stdout:2722 clone 21 order 18 su 1024 sc 4 2024-09-21T15:33:36.207 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.207 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.222 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:36.278 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:36.274+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:36.379 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:36.374+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:36.389 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.389 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:36.480 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:36.475+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:36.523 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.523 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.537 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:36.581 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:36.579+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:36.682 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:36.678+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:36.705 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.705 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.719 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:36.779 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:33:36.783 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:36.778+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:36.844 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.844 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:36.904 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:36.903+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:36.905 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone20 from 0xe24c77d (overlap 0x69d081) to 0xa94d9 2024-09-21T15:33:36.982 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:36.982 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:36.985 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:36.982+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:36.997 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:37.002 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T15:33:37.086 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:37.086+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:37.088 INFO:teuthology.orchestra.run.smithi107.stdout:2723 read 0xdb08902 thru 0xdb1242e (0x9b2d bytes) 2024-09-21T15:33:37.089 INFO:teuthology.orchestra.run.smithi107.stdout:2724 read 0xa9b060a thru 0xa9b3935 (0x332c bytes) 2024-09-21T15:33:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:2725 trunc from 0xe24c77d to 0x79a7829 2024-09-21T15:33:37.126 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.127 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:37.140 INFO:teuthology.orchestra.run.smithi107.stdout:2727 trunc from 0x79a7829 to 0xb07110d 2024-09-21T15:33:37.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:37.153 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.154 INFO:teuthology.orchestra.run.smithi107.stdout:2729 read 0xa2d4672 thru 0xa2da0b5 (0x5a44 bytes) 2024-09-21T15:33:37.154 INFO:teuthology.orchestra.run.smithi107.stdout:2730 writesame 0xf210f5 thru 0xf2848c data_size 0xc(0x7398 bytes) 2024-09-21T15:33:37.169 INFO:teuthology.orchestra.run.smithi107.stdout:2732 read 0x9aaf95c thru 0x9aba811 (0xaeb6 bytes) 2024-09-21T15:33:37.183 INFO:teuthology.orchestra.run.smithi107.stdout:2733 writesame 0xbc58951 thru 0xbc5b8aa data_size 0xb(0x2f5a bytes) 2024-09-21T15:33:37.183 INFO:teuthology.orchestra.run.smithi107.stdout:2734 read 0x8e1ed6b thru 0x8e2741b (0x86b1 bytes) 2024-09-21T15:33:37.187 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:37.183+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:37.194 INFO:teuthology.orchestra.run.smithi107.stdout:2735 write 0xbd11179 thru 0xbd1b75c (0xa5e4 bytes) 2024-09-21T15:33:37.207 INFO:teuthology.orchestra.run.smithi107.stdout:2737 read 0x486fe0f thru 0x4876c91 (0x6e83 bytes) 2024-09-21T15:33:37.207 INFO:teuthology.orchestra.run.smithi107.stdout:2738 read 0xa174e4 thru 0xa225cb (0xb0e8 bytes) 2024-09-21T15:33:37.208 INFO:teuthology.orchestra.run.smithi107.stdout:2739 write 0x2ea729d thru 0x2eadeba (0x6c1e bytes) 2024-09-21T15:33:37.222 INFO:teuthology.orchestra.run.smithi107.stdout:2741 read 0x696d6e3 thru 0x6971a42 (0x4360 bytes) 2024-09-21T15:33:37.223 INFO:teuthology.orchestra.run.smithi107.stdout:2742 read 0x619a0c6 thru 0x61a21f4 (0x812f bytes) 2024-09-21T15:33:37.236 INFO:teuthology.orchestra.run.smithi107.stdout:2743 read 0x2bc301d thru 0x2bcd586 (0xa56a bytes) 2024-09-21T15:33:37.237 INFO:teuthology.orchestra.run.smithi107.stdout:2745 read 0x8457ec6 thru 0x8465fbb (0xe0f6 bytes) 2024-09-21T15:33:37.237 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.237 INFO:teuthology.orchestra.run.smithi107.stdout:2747 read 0x1596b8d thru 0x15a5f92 (0xf406 bytes) 2024-09-21T15:33:37.237 INFO:teuthology.orchestra.run.smithi107.stdout:2748 punch from 0x4d60992 to 0x4d6809d, (0x770b bytes) 2024-09-21T15:33:37.237 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.238 INFO:teuthology.orchestra.run.smithi107.stdout:2750 writesame 0x53bfccc thru 0x53c9162 data_size 0x1(0x9497 bytes) 2024-09-21T15:33:37.265 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.265 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:37.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:37.324 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:37.323+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:37.389 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:37.387+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:37.449 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.449 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:37.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:37.508 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:37.507+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:37.591 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:37.587+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:2751 writesame 0xc6361fb thru 0xc63e183 data_size 0x1(0x7f89 bytes) 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:2755 read 0x83c07ce thru 0x83cc58f (0xbdc2 bytes) 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:2757 read 0x8e26a8f thru 0x8e2c7b8 (0x5d2a bytes) 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.645 INFO:teuthology.orchestra.run.smithi107.stdout:2759 write 0x403bf8f thru 0x403e966 (0x29d8 bytes) 2024-09-21T15:33:37.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:37.660 INFO:teuthology.orchestra.run.smithi107.stdout:2760 read 0x839f5b7 thru 0x83a7a63 (0x84ad bytes) 2024-09-21T15:33:37.661 INFO:teuthology.orchestra.run.smithi107.stdout:2761 punch from 0x604df38 to 0x605d3dd, (0xf4a5 bytes) 2024-09-21T15:33:37.664 INFO:teuthology.orchestra.run.smithi107.stdout:2762 trunc from 0xc63e184 to 0xa96437e 2024-09-21T15:33:37.689 INFO:teuthology.orchestra.run.smithi107.stdout:2763 trunc from 0xa96437e to 0x8aa6fed 2024-09-21T15:33:37.691 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:37.687+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:37.701 INFO:teuthology.orchestra.run.smithi107.stdout:2764 write 0x69e41f2 thru 0x69e51d7 (0xfe6 bytes) 2024-09-21T15:33:37.715 INFO:teuthology.orchestra.run.smithi107.stdout:2765 read 0x4b530c7 thru 0x4b537db (0x715 bytes) 2024-09-21T15:33:37.715 INFO:teuthology.orchestra.run.smithi107.stdout:2766 punch from 0x64fd32f to 0x64fe49c, (0x116d bytes) 2024-09-21T15:33:37.731 INFO:teuthology.orchestra.run.smithi107.stdout:2769 write 0xc39491d thru 0xc39dc71 (0x9355 bytes) 2024-09-21T15:33:37.746 INFO:teuthology.orchestra.run.smithi107.stdout:2770 punch from 0x10990de to 0x109a88c, (0x17ae bytes) 2024-09-21T15:33:37.746 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.746 INFO:teuthology.orchestra.run.smithi107.stdout:2773 write 0x7100d19 thru 0x7109712 (0x89fa bytes) 2024-09-21T15:33:37.762 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:37.762 INFO:teuthology.orchestra.run.smithi107.stdout:2775 punch from 0x9ca24d1 to 0x9caeb30, (0xc65f bytes) 2024-09-21T15:33:37.762 INFO:teuthology.orchestra.run.smithi107.stdout:2776 write 0xbdf10 thru 0xca256 (0xc347 bytes) 2024-09-21T15:33:37.792 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:37.790+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:37.795 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.795 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:37.811 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:37.893 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:37.890+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:37.994 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:37.991+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:37.998 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:37.998 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:38.013 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:38.016 INFO:teuthology.orchestra.run.smithi107.stdout:2778 write 0x937f66c thru 0x9386eae (0x7843 bytes) 2024-09-21T15:33:38.029 INFO:teuthology.orchestra.run.smithi107.stdout:2780 write 0x4fd549a thru 0x4fe51cc (0xfd33 bytes) 2024-09-21T15:33:38.037 INFO:teuthology.orchestra.run.smithi107.stdout:2781 punch from 0x19ab4a9 to 0x19b6cb4, (0xb80b bytes) 2024-09-21T15:33:38.037 INFO:teuthology.orchestra.run.smithi107.stdout:2782 read 0xa6b9eda thru 0xa6c8e4b (0xef72 bytes) 2024-09-21T15:33:38.038 INFO:teuthology.orchestra.run.smithi107.stdout:2783 read 0xa66d35e thru 0xa676b8a (0x982d bytes) 2024-09-21T15:33:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:2784 write 0xd981dd1 thru 0xd98bdcd (0x9ffd bytes) 2024-09-21T15:33:38.057 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:38.057 INFO:teuthology.orchestra.run.smithi107.stdout:2792 write 0xc3abc83 thru 0xc3ba63e (0xe9bc bytes) 2024-09-21T15:33:38.060 INFO:teuthology.orchestra.run.smithi107.stdout:2793 punch from 0x18cf913 to 0x18de90b, (0xeff8 bytes) 2024-09-21T15:33:38.060 INFO:teuthology.orchestra.run.smithi107.stdout:2794 punch from 0x7c37928 to 0x7c39778, (0x1e50 bytes) 2024-09-21T15:33:38.060 INFO:teuthology.orchestra.run.smithi107.stdout:2795 writesame 0x897fcfb thru 0x898f9f2 data_size 0x14(0xfcf8 bytes) 2024-09-21T15:33:38.068 INFO:teuthology.orchestra.run.smithi107.stdout:2798 read 0x9019bd9 thru 0x90288fd (0xed25 bytes) 2024-09-21T15:33:38.068 INFO:teuthology.orchestra.run.smithi107.stdout:2799 write 0x5e25909 thru 0x5e34708 (0xee00 bytes) 2024-09-21T15:33:38.076 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:38.076 INFO:teuthology.orchestra.run.smithi107.stdout:2801 clone 22 order 19 su 32768 sc 9 2024-09-21T15:33:38.095 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:38.091+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:38.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:38.153 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:38.154 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:38.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:38.196 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:38.195+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:38.280 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:38.280 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:38.289 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:38.289 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:38.296 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:38.296 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:38.296 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:38.294+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:38.303 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:38.313 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:38.313 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:38.329 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:38.330 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:38.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:38.348 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:38.366 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:38.366 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:38.383 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:38.383 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:38.397 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:38.395+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:38.401 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:38.402 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:776 3.1a 3:5f904e79:::rbd_data.11911fd826fd.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.794905427000003, 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012047844, 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 776 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675576+0000", 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675574+0000", 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:38.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675584+0000", 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9480000000000003e-06 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675605+0000", 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1052000000000001e-05 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675615+0000", 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8870000000000004e-06 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675673+0000", 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7797000000000002e-05 2024-09-21T15:33:38.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.675827+0000", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015415700000000001 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.678066+0000", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022387489999999999 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687531+0000", 2024-09-21T15:33:38.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094649780000000006 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687563+0000", 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2053e-05 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687599+0000", 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6103999999999999e-05 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.687624+0000", 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1276 3.1d e27/21 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head v 27'7, mlcod=26'5)", 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 44.678808074000003, 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.030724027000000001, 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791673+0000", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.791671+0000", 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:38.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800055+0000", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083839020000000004 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800095+0000", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0732999999999998e-05 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800112+0000", 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:38.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800235+0000", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012232899999999999 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.800273+0000", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8287000000000001e-05 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822318+0000", 2024-09-21T15:33:38.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022044713000000001 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822397+0000", 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9556999999999997e-05 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1279 3.1d e27/21 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head v 27'8, mlcod=26'5)", 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:38.483 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 44.64222796, 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021724157000000001, 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828253+0000", 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.828250+0000", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833754+0000", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055040169999999999 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833802+0000", 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8226000000000002e-05 2024-09-21T15:33:38.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833828+0000", 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5714000000000001e-05 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.833954+0000", 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125726 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.834027+0000", 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2887999999999997e-05 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849886+0000", 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015859807 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.849978+0000", 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1261999999999995e-05 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1280 3.1d e27/21 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 44.623299265999997, 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022280950000000001, 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847182+0000", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.847180+0000", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856420+0000", 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092405760000000003 2024-09-21T15:33:38.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856462+0000", 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1594999999999997e-05 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856482+0000", 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0682e-05 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856604+0000", 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122037 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.856664+0000", 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9619e-05 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869381+0000", 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012716794 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869463+0000", 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2262000000000006e-05 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3973 3.e e31/21 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head v 31'18, mlcod=28'15)", 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 39.883550235000001, 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039812899999999998, 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586931+0000", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.586928+0000", 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597019+0000", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010090432 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597063+0000", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4409999999999997e-05 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597082+0000", 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8606000000000001e-05 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597244+0000", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000162635 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.597305+0000", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1316999999999996e-05 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626653+0000", 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029347326 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.626744+0000", 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1260000000000004e-05 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4210 3.1 e31/21 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head v 31'22, mlcod=29'19)", 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 39.748327842000002, 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.039204640999999998, 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.496 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722154+0000", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722152+0000", 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731550+0000", 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093980739999999993 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731596+0000", 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6452999999999997e-05 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731614+0000", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7657000000000001e-05 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731744+0000", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012995199999999999 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.731809+0000", 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5296000000000006e-05 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761237+0000", 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029427707000000001 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.761358+0000", 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121552 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:4208 3.11 e31/21 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head v 31'75, mlcod=29'73)", 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 39.745299111999998, 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.037828334999999998, 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725182+0000", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.725179+0000", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737821+0000", 2024-09-21T15:33:38.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012642229 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737851+0000", 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9819999999999999e-05 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737868+0000", 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6552999999999999e-05 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737939+0000", 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0945999999999995e-05 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.737975+0000", 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6497e-05 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762948+0000", 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024972636999999999 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763011+0000", 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2992999999999998e-05 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:5584 3.1a 3:5d680a0d:::rbd_data.11917ba4f9b.0000000000000022:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.447093778999999, 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071935658, 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 5584 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023388+0000", 2024-09-21T15:33:38.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023385+0000", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023399+0000", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3933e-05 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023430+0000", 2024-09-21T15:33:38.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0647e-05 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023444+0000", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4406999999999999e-05 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023564+0000", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120055 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.023796+0000", 2024-09-21T15:33:38.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000231695 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.037292+0000", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013496006 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095215+0000", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057922545999999998 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095227+0000", 2024-09-21T15:33:38.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2547e-05 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095279+0000", 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2163000000000002e-05 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095323+0000", 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3859000000000001e-05 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:5584 3.1a e33/21 ondisk, result = 0)", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.396508142000002, 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021245146999999999, 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073973+0000", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073971+0000", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.073983+0000", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1783999999999999e-05 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074016+0000", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3386000000000002e-05 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.074032+0000", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6554000000000001e-05 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095197+0000", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021164130999999999 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095207+0000", 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0601999999999999e-05 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:02.095218+0000", 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1224e-05 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10849 3.1a 3:5e4725af:::rbd_data.11917b2c111e.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:38.514 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.034797677, 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019049979000000002, 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10849 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435684+0000", 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435682+0000", 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435694+0000", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1625e-05 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435718+0000", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4127000000000001e-05 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435730+0000", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.202e-05 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.435822+0000", 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2070000000000004e-05 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.436008+0000", 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000186286 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.438479+0000", 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024705510000000001 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454450+0000", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015971078 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454530+0000", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0412000000000002e-05 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454617+0000", 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6753000000000001e-05 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:12.454734+0000", 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116772 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16689 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4732280930000003, 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099936150000000008, 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.520 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16689 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997253+0000", 2024-09-21T15:33:38.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997252+0000", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997257+0000", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8420000000000004e-06 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997272+0000", 2024-09-21T15:33:38.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5102e-05 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997284+0000", 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1616e-05 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006897+0000", 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096132590000000007 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007002+0000", 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010490200000000001 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007247+0000", 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024475099999999997 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16690 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4731873359999996, 2024-09-21T15:33:38.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010305910999999999, 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16690 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997294+0000", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997293+0000", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997300+0000", 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8170000000000001e-06 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997315+0000", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4793e-05 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997327+0000", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1953000000000001e-05 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007284+0000", 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099574629999999997 2024-09-21T15:33:38.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007388+0000", 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000103568 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007600+0000", 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000212143 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16692 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4731016859999997, 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01059863, 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16692 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997380+0000", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997379+0000", 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997385+0000", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3820000000000001e-06 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997400+0000", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5162e-05 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997412+0000", 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1749999999999999e-05 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007635+0000", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010222663 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007732+0000", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.7085999999999998e-05 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007978+0000", 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024659200000000001 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16694 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4730101869999999, 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010846381, 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16694 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.533 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997471+0000", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997470+0000", 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997475+0000", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9429999999999999e-06 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997490+0000", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5285e-05 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997502+0000", 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.199e-05 2024-09-21T15:33:38.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008013+0000", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010510733 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008114+0000", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000100599 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008318+0000", 2024-09-21T15:33:38.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020389400000000001 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16696 3.15 3:aa19fb9b:::rbd_data.1191dfb05674.0000000000000003:16 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4729266130000003, 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011432552, 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.537 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16696 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997555+0000", 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997554+0000", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997559+0000", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.994e-06 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997576+0000", 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6767000000000001e-05 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997587+0000", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1810999999999999e-05 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008339+0000", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010751805 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008417+0000", 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7960000000000006e-05 2024-09-21T15:33:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008987+0000", 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00057021800000000005 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16697 3.1a 3:5bcd11c0:::rbd_data.1191bd191516.0000000000000008:14 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4728838160000004, 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010924658, 2024-09-21T15:33:38.541 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16697 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997598+0000", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997597+0000", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997602+0000", 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9620000000000001e-06 2024-09-21T15:33:38.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.997618+0000", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6535999999999999e-05 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008249+0000", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010630707 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008319+0000", 2024-09-21T15:33:38.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0352999999999995e-05 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008403+0000", 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3627000000000004e-05 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008522+0000", 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119447 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16790 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4616985639999998, 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098152529999999995, 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16790 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.546 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008783+0000", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008782+0000", 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:38.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008786+0000", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6569999999999999e-06 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008796+0000", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8199999999999992e-06 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008803+0000", 2024-09-21T15:33:38.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8609999999999998e-06 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018427+0000", 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096234849999999993 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018499+0000", 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1666000000000001e-05 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018598+0000", 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.946e-05 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16793 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4616708520000001, 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099596430000000007, 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16793 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008811+0000", 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008810+0000", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008817+0000", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4189999999999996e-06 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008827+0000", 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008835+0000", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8099999999999998e-06 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018630+0000", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097946439999999999 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018683+0000", 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3457e-05 2024-09-21T15:33:38.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018770+0000", 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6965999999999994e-05 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16796 3.15 3:a93f0e12:::rbd_data.1191bd191516.0000000000000007:14 [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4616386969999997, 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010146664, 2024-09-21T15:33:38.554 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16796 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008843+0000", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008842+0000", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008846+0000", 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5820000000000002e-06 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008856+0000", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0183000000000001e-05 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008866+0000", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8740000000000007e-06 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018820+0000", 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099546500000000007 2024-09-21T15:33:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018894+0000", 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3609000000000006e-05 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.018989+0000", 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5456000000000002e-05 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:17308 3.1a 3:5e234bb8:::rbd_data.1191c93bd12d.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.4368538769999999, 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012577608000000001, 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 17308 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033628+0000", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.033627+0000", 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034352+0000", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072501 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034367+0000", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5798e-05 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034375+0000", 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6320000000000008e-06 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034583+0000", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020768800000000001 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.034802+0000", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021902600000000001 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.035066+0000", 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026464199999999999 2024-09-21T15:33:38.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.036863+0000", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001796834 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046146+0000", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092828340000000002 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046176+0000", 2024-09-21T15:33:38.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9972999999999999e-05 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.046205+0000", 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9161000000000001e-05 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:38.564 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:38.566 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:38.566 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:38.566+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:38.666 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:38.663+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:38.735 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:38.735 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:38.735 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:38.735 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:38.751 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:38.768 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:38.763+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:38.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:33:38.869 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:38.867+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:38.882 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:38.883 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:38.883 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:38.883 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:38.883 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:38.897 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:38.925 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone21 from 0xd98bdce (overlap 0x79a7829) to 0x6d7d28b 2024-09-21T15:33:38.969 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:38.967+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:39.034 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 62.244683807999998, 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:39.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:39.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:39.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 60.238072602000003, 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:39.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:39.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.663244882999997, 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.047 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:39.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:872 3.5 3:a31b1cfe:::rbd_data.11911fd826fd.0000000000000008:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.038972436000002, 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0076432899999999996, 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 872 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993096+0000", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993095+0000", 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:39.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993106+0000", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1588e-05 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993118+0000", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1914e-05 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993127+0000", 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7059999999999993e-06 2024-09-21T15:33:39.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993260+0000", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013325199999999999 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.993527+0000", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000266464 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.994037+0000", 2024-09-21T15:33:39.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00051008300000000004 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.996466+0000", 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024291550000000001 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998600+0000", 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0021341989999999998 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.998759+0000", 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000158269 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.000739+0000", 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0019806120000000001 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:989 3.c 3:35c0a0a7:::rbd_data.11911fd826fd.0000000000000037:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:39.058 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.994686862999998, 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007642383, 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 989 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037381+0000", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037379+0000", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037392+0000", 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2522e-05 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037411+0000", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9052e-05 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037424+0000", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2822e-05 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037498+0000", 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3934000000000001e-05 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037639+0000", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014103499999999999 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.037876+0000", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236782 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.039324+0000", 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001448392 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044959+0000", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056346479999999999 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.044994+0000", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5728000000000001e-05 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.045024+0000", 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9522e-05 2024-09-21T15:33:39.063 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.361430890000001, 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:39.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:39.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.069 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282614236000001, 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:39.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:39.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:39.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.252261222999998, 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:39.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:39.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:39.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:39.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.244389527999999, 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:39.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:39.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:39.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.233210079999999, 2024-09-21T15:33:39.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:39.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.227977713999998, 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.090 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:39.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:39.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.209492433000001, 2024-09-21T15:33:39.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.082338931000002, 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:39.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.102 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.103 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.082182609, 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.104 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:39.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:39.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:39.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.109 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.076500383999999, 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:39.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:39.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.075537564999998, 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:39.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.314692642000001, 2024-09-21T15:33:39.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:39.124 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.005668008000001, 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.125 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:39.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.726603031, 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.131 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:39.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:39.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:39.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 22.395763534, 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:39.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.142 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.143 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:39.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:39.144 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:39.143+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:39.163 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T15:33:39.177 INFO:teuthology.orchestra.run.smithi107.stdout:2802 trunc from 0xd98bdce to 0x7a83f5b 2024-09-21T15:33:39.210 INFO:teuthology.orchestra.run.smithi107.stdout:2803 punch from 0x47c5f93 to 0x47cb61b, (0x5688 bytes) 2024-09-21T15:33:39.210 INFO:teuthology.orchestra.run.smithi107.stdout:2806 writesame 0x7479ea2 thru 0x747b7b2 data_size 0x1f(0x1911 bytes) 2024-09-21T15:33:39.230 INFO:teuthology.orchestra.run.smithi107.stdout:2807 write 0x9295813 thru 0x92a286c (0xd05a bytes) 2024-09-21T15:33:39.240 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:39.240 INFO:teuthology.orchestra.run.smithi107.stdout:2809 flatten 2024-09-21T15:33:39.247 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:39.243+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:39.278 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:39.278 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:39.278 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:39.278 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:39.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:39.346 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:39.346+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:39.411 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:39.411 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:39.411 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:39.411 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:39.411 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:39.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:39.446 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:39.443+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:39.548 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:39.550+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.128422497999999, 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.593 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.898145121999999, 2024-09-21T15:33:39.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:39.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.602 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.596774330999999, 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.921260511, 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.592404149, 2024-09-21T15:33:39.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:39.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5923876569999997, 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.617 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:39.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5923712850000005, 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.621 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:39.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:39.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5923546200000001, 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.625 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.592337798, 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.629 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:39.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:39.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.59200988, 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.633 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:39.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:39.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:39.637 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5919792380000004, 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5919610940000002, 2024-09-21T15:33:39.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:39.645 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.591930638, 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:39.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.649 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5919123500000003, 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:39.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:39.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:39.653 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5918950619999999, 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5918783960000003, 2024-09-21T15:33:39.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:39.661 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5918540490000002, 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:39.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.665 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5918379070000004, 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:39.669 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5918213620000001, 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.673 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 5.5917938180000002, 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:39.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:39.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:39.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.677 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:39.678 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:39.678 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:39.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:39.679 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:39.678+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:39.780 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:39.778+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:39.808 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:33:39.846 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:39.846 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:39.846 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:39.846 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:39.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:39.904 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:39.906+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:39.982 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:39.982+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:40.030 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:40.031 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:40.031 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:40.031 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:40.031 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:40.045 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:40.084 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:40.079+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:40.186 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:40.182+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.227 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4198.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 69.158778919, 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016190264999999999, 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4198", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.170:0/2212408644", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 117 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064453+0000", 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064452+0000", 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064459+0000", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.703e-06 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064468+0000", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8969999999999997e-06 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.064475+0000", 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9600000000000003e-06 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078697+0000", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014221981 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.078819+0000", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012227599999999999 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 0", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.079009+0000", 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019024800000000001 2024-09-21T15:33:40.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080242+0000", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0012331309999999999 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080530+0000", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028745700000000002 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080597+0000", 2024-09-21T15:33:40.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6649999999999994e-05 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:31.080643+0000", 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6569000000000001e-05 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1190 3.11 3:8bc4bd9b:::rbd_data.1191bb6b225c.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.710305990999998, 2024-09-21T15:33:40.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022028537000000001, 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1190 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512926+0000", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512924+0000", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512937+0000", 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3223e-05 2024-09-21T15:33:40.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512966+0000", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9215999999999999e-05 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.512980+0000", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4273e-05 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513106+0000", 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012556000000000001 2024-09-21T15:33:40.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.513349+0000", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000243341 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.517115+0000", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003766138 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534677+0000", 2024-09-21T15:33:40.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017561961000000001 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534756+0000", 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8382999999999998e-05 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534846+0000", 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0075999999999996e-05 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.534954+0000", 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108446 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1220 3.7 e27/21 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head v 27'13, mlcod=27'11)", 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.623073040999998, 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035610309, 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600159+0000", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.600156+0000", 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615711+0000", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015554596 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615756+0000", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5343000000000003e-05 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615775+0000", 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8564e-05 2024-09-21T15:33:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615881+0000", 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010644199999999999 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.615921+0000", 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9133e-05 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635654+0000", 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019733581 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.635769+0000", 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0001149 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.243 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1278 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.438678748000001, 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.060972842999999999, 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1278 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784551+0000", 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801400+0000", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01684892 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801431+0000", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1581999999999997e-05 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801451+0000", 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.948e-05 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801589+0000", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013792500000000001 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801994+0000", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040495700000000001 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.802913+0000", 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00091958600000000001 2024-09-21T15:33:40.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.832473+0000", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029559932000000001 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845354+0000", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012880796999999999 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845448+0000", 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3746000000000007e-05 2024-09-21T15:33:40.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845526+0000", 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7972999999999994e-05 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3973 3.e 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11281964~20966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.655085751000001, 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.059230786000000001, 2024-09-21T15:33:40.249 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3973 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568144+0000", 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583870+0000", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015726602999999999 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583922+0000", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1239000000000002e-05 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583942+0000", 2024-09-21T15:33:40.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0213000000000001e-05 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584121+0000", 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000179469 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584383+0000", 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026151100000000002 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.585420+0000", 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0010370189999999999 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.625790+0000", 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040369751000000002 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627101+0000", 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001311127 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627255+0000", 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015414899999999999 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627377+0000", 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122009 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4208 3.11 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:40.255 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.513751069000001, 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054071906000000003, 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4208 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:40.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709478+0000", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721564+0000", 2024-09-21T15:33:40.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012085933 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721605+0000", 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0862999999999998e-05 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721625+0000", 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9953e-05 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721772+0000", 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014629199999999999 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722036+0000", 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000264415 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.723537+0000", 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015015549999999999 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762221+0000", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038683842000000003 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763323+0000", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011017150000000001 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763453+0000", 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130199 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763553+0000", 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9382000000000005e-05 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14128 3.11 3:8a0b4402:::rbd_data.1191bd191516.000000000000003e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 15.607616162999999, 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045981519999999998, 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:40.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14128 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615614+0000", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615629+0000", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4788e-05 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615645+0000", 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6401000000000001e-05 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615654+0000", 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6160000000000002e-06 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615738+0000", 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4012999999999997e-05 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615954+0000", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021574100000000001 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.625676+0000", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097218289999999995 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661339+0000", 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035663551000000002 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661449+0000", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109675 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661536+0000", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7069000000000003e-05 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661597+0000", 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1199999999999997e-05 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16747 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 319488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2333499290000001, 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016176724999999999, 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16747 2024-09-21T15:33:40.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989882+0000", 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989881+0000", 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989884+0000", 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.356e-06 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989890+0000", 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6640000000000003e-06 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989894+0000", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3749999999999996e-06 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.005838+0000", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015944100999999999 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.005947+0000", 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000108707 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006059+0000", 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011188000000000001 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16749 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 335872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2333336599999996, 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016557470000000001, 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.271 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16749 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989898+0000", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989900+0000", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1129999999999999e-06 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989906+0000", 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.609e-06 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989910+0000", 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4050000000000002e-06 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006159+0000", 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016249257999999999 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006314+0000", 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154869 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006456+0000", 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014155299999999999 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16751 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.275 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.233317746, 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01678932, 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16751 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:40.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989914+0000", 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989916+0000", 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.114e-06 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989922+0000", 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8950000000000003e-06 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989926+0000", 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006503+0000", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016576911 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006602+0000", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9429e-05 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006703+0000", 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010092 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16753 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2333014909999997, 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017122808999999999, 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16753 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989930+0000", 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989932+0000", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3029999999999998e-06 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989938+0000", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0009999999999998e-06 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989943+0000", 2024-09-21T15:33:40.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.578e-06 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006767+0000", 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.016823683999999998 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.006915+0000", 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014854899999999999 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007053+0000", 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013799600000000001 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16755 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2332848959999998, 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017350775999999998, 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16755 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989947+0000", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989949+0000", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.216e-06 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989955+0000", 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:33:40.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989959+0000", 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5709999999999999e-06 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007125+0000", 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017165369999999999 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007218+0000", 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3270999999999995e-05 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007298+0000", 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9561999999999995e-05 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16758 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 401408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:40.288 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2332472340000002, 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019563328000000001, 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16758 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989985+0000", 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989984+0000", 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989986+0000", 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0810000000000001e-06 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989992+0000", 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.789e-06 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.989996+0000", 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.374e-06 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007352+0000", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017355085999999999 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007494+0000", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000142936 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009548+0000", 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020534360000000001 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16760 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2332146829999999, 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017899899, 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16760 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.293 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990017+0000", 2024-09-21T15:33:40.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990019+0000", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1720000000000001e-06 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990025+0000", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7989999999999999e-06 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990029+0000", 2024-09-21T15:33:40.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4120000000000003e-06 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007660+0000", 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017630959000000002 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007804+0000", 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014346000000000001 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007917+0000", 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011346899999999999 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16762 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2331819509999997, 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018138254999999999, 2024-09-21T15:33:40.297 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16762 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990050+0000", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990049+0000", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990052+0000", 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.881e-06 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990058+0000", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9730000000000002e-06 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990063+0000", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6990000000000002e-06 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.007955+0000", 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017891503999999999 2024-09-21T15:33:40.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008077+0000", 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122077 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008188+0000", 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111501 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16763 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2331645260000004, 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018389483000000002, 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16763 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990067+0000", 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990069+0000", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0959999999999999e-06 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990075+0000", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1519999999999997e-06 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990080+0000", 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5159999999999996e-06 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008226+0000", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01814613 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008342+0000", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011653499999999999 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008457+0000", 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114404 2024-09-21T15:33:40.305 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16765 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2331341680000003, 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018629436999999999, 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16765 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990098+0000", 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990097+0000", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990099+0000", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1849999999999998e-06 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990105+0000", 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9649999999999996e-06 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990110+0000", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6040000000000003e-06 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008495+0000", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018384482000000001 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008618+0000", 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012336700000000001 2024-09-21T15:33:40.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008727+0000", 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109144 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16767 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.233084571, 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018847929999999999, 2024-09-21T15:33:40.310 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16767 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990147+0000", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990149+0000", 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.323e-06 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990155+0000", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8479999999999997e-06 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990160+0000", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6419999999999998e-06 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008763+0000", 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018603831000000001 2024-09-21T15:33:40.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008884+0000", 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120865 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.008995+0000", 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000110802 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16769 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2330518689999996, 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019085418, 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16769 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990180+0000", 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990186+0000", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4989999999999999e-06 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990193+0000", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.849e-06 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990198+0000", 2024-09-21T15:33:40.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8760000000000004e-06 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009032+0000", 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018834178 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009154+0000", 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012181299999999999 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009265+0000", 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011154000000000001 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16770 3.d 3:b29a440c:::rbd_data.1191dfb05674.0000000000000005:16 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 6.2330295659999999, 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019331745000000001, 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:40.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16770 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990202+0000", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990204+0000", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.198e-06 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990209+0000", 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9490000000000004e-06 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.990212+0000", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5629999999999999e-06 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009301+0000", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019089565999999999 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009420+0000", 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119072 2024-09-21T15:33:40.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.009534+0000", 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113731 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:40.323 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:40.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:40.329 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:40.323+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:40.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:40.425 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:40.423+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:40.483 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:40.484 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:40.484 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:40.484 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:40.494 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:40.494 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:40.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:33:40.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:40.527 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:40.526+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:40.614 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:40.616 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:40.627 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:40.626+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:40.636 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:40.637 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:40.660 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:40.660 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:40.694 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:40.694 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:40.712 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:40.712 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:40.727 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:40.723+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:40.729 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:40.729 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:40.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:40.765 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:40.765 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:40.795 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:40.795 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:40.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:40.822 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:40.827 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:40.823+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:40.922 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:40.923 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:40.929 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:40.930+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:40.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:40.956 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:33:40.977 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-21T15:33:39.803046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":513980134,"num_objects":800,"num_object_clones":166,"num_object_copies":1600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":800,"num_whiteouts":353,"num_read":38678,"num_read_kb":116940,"num_write":6933,"num_write_kb":165240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":5198342,"num_keys_recovered":35,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5693,"ondisk_log_size":5693,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":369840,"kb_used_data":84044,"kb_used_omap":12,"kb_used_meta":285683,"kb_avail":754604880,"statfs":{"total":773094113280,"available":772715397120,"internally_reserved":0,"allocated":86061056,"data_stored":98114967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":292539961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51480639,"num_objects":133,"num_object_clones":23,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":29,"num_read":7948,"num_read_kb":27857,"num_write":1026,"num_write_kb":16653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.035316"},"pg_stats":[{"pgid":"3.1f","version":"63'63","reported_seq":195,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.759613+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:33:37.759613+0000","last_peered":"2024-09-21T15:33:37.759613+0000","last_clean":"2024-09-21T15:33:37.759613+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:33:37.759613+0000","last_undegraded":"2024-09-21T15:33:37.759613+0000","last_fullsized":"2024-09-21T15:33:37.759613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":16304504,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":118,"num_read_kb":1734,"num_write":59,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'206","reported_seq":506,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.223671+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:33:37.223671+0000","last_peered":"2024-09-21T15:33:37.223671+0000","last_clean":"2024-09-21T15:33:37.223671+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:33:37.223671+0000","last_undegraded":"2024-09-21T15:33:37.223671+0000","last_fullsized":"2024-09-21T15:33:37.223671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":17516087,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":914,"num_read_kb":7829,"num_write":329,"num_write_kb":22553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"63'66","reported_seq":208,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.662573+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:33:37.662573+0000","last_peered":"2024-09-21T15:33:37.662573+0000","last_clean":"2024-09-21T15:33:37.662573+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:33:37.662573+0000","last_undegraded":"2024-09-21T15:33:37.662573+0000","last_fullsized":"2024-09-21T15:33:37.662573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":12750223,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":127,"num_read_kb":979,"num_write":80,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'113","reported_seq":852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.745040+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:33:37.745040+0000","last_peered":"2024-09-21T15:33:37.745040+0000","last_clean":"2024-09-21T15:33:37.745040+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:33:37.745040+0000","last_undegraded":"2024-09-21T15:33:37.745040+0000","last_fullsized":"2024-09-21T15:33:37.745040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":41546130,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":809,"num_read_kb":1171,"num_write":110,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"63'194","reported_seq":739,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:36.778942+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:33:36.778942+0000","last_peered":"2024-09-21T15:33:36.778942+0000","last_clean":"2024-09-21T15:33:36.778942+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:33:36.778942+0000","last_undegraded":"2024-09-21T15:33:36.778942+0000","last_fullsized":"2024-09-21T15:33:36.778942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":8348012,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":1238,"num_read_kb":2947,"num_write":197,"num_write_kb":1778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'97","reported_seq":1262,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.048579+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:33:38.048579+0000","last_peered":"2024-09-21T15:33:38.048579+0000","last_clean":"2024-09-21T15:33:38.048579+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:33:38.048579+0000","last_undegraded":"2024-09-21T15:33:38.048579+0000","last_fullsized":"2024-09-21T15:33:38.048579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":7631140,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":1163,"num_read_kb":4178,"num_write":86,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"63'359","reported_seq":1784,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.009866+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:33:38.009866+0000","last_peered":"2024-09-21T15:33:38.009866+0000","last_clean":"2024-09-21T15:33:38.009866+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:33:38.009866+0000","last_undegraded":"2024-09-21T15:33:38.009866+0000","last_fullsized":"2024-09-21T15:33:38.009866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":39532976,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":2794,"num_read_kb":2432,"num_write":518,"num_write_kb":2962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'323","reported_seq":608,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:36.778835+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:33:36.778835+0000","last_peered":"2024-09-21T15:33:36.778835+0000","last_clean":"2024-09-21T15:33:36.778835+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:33:36.778835+0000","last_undegraded":"2024-09-21T15:33:36.778835+0000","last_fullsized":"2024-09-21T15:33:36.778835+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":6918971,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1821,"num_read_kb":1551,"num_write":483,"num_write_kb":9868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"63'125","reported_seq":810,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.167468+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:33:37.167468+0000","last_peered":"2024-09-21T15:33:37.167468+0000","last_clean":"2024-09-21T15:33:37.167468+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:33:37.167468+0000","last_undegraded":"2024-09-21T15:33:37.167468+0000","last_fullsized":"2024-09-21T15:33:37.167468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":22029455,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":933,"num_read_kb":5924,"num_write":178,"num_write_kb":3566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"63'230","reported_seq":1299,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.171683+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:33:37.171683+0000","last_peered":"2024-09-21T15:33:37.171683+0000","last_clean":"2024-09-21T15:33:37.171683+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:33:37.171683+0000","last_undegraded":"2024-09-21T15:33:37.171683+0000","last_fullsized":"2024-09-21T15:33:37.171683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":23178028,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":2545,"num_read_kb":4076,"num_write":222,"num_write_kb":1825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"63'140","reported_seq":332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.024863+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:33:37.024863+0000","last_peered":"2024-09-21T15:33:37.024863+0000","last_clean":"2024-09-21T15:33:37.024863+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:33:37.024863+0000","last_undegraded":"2024-09-21T15:33:37.024863+0000","last_fullsized":"2024-09-21T15:33:37.024863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":4346145,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":424,"num_read_kb":4035,"num_write":172,"num_write_kb":2227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"63'317","reported_seq":1214,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.075599+0000","last_change":"2024-09-21T15:32:48.726261+0000","last_active":"2024-09-21T15:33:38.075599+0000","last_peered":"2024-09-21T15:33:38.075599+0000","last_clean":"2024-09-21T15:33:38.075599+0000","last_became_active":"2024-09-21T15:32:46.731389+0000","last_became_peered":"2024-09-21T15:32:46.731389+0000","last_unstale":"2024-09-21T15:33:38.075599+0000","last_undegraded":"2024-09-21T15:33:38.075599+0000","last_fullsized":"2024-09-21T15:33:38.075599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:02.678176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":11996929,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2798,"num_read_kb":3084,"num_write":381,"num_write_kb":1353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.807064+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:33:34.807064+0000","last_peered":"2024-09-21T15:33:34.807064+0000","last_clean":"2024-09-21T15:33:34.807064+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:33:34.807064+0000","last_undegraded":"2024-09-21T15:33:34.807064+0000","last_fullsized":"2024-09-21T15:33:34.807064+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"63'171","reported_seq":1179,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.049268+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:33:37.049268+0000","last_peered":"2024-09-21T15:33:37.049268+0000","last_clean":"2024-09-21T15:33:37.049268+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:33:37.049268+0000","last_undegraded":"2024-09-21T15:33:37.049268+0000","last_fullsized":"2024-09-21T15:33:37.049268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":11472147,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2463,"num_read_kb":4770,"num_write":157,"num_write_kb":1720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.774538+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:33:34.774538+0000","last_peered":"2024-09-21T15:33:34.774538+0000","last_clean":"2024-09-21T15:33:34.774538+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:33:34.774538+0000","last_undegraded":"2024-09-21T15:33:34.774538+0000","last_fullsized":"2024-09-21T15:33:34.774538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"63'89","reported_seq":618,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.057489+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:33:38.057489+0000","last_peered":"2024-09-21T15:33:38.057489+0000","last_clean":"2024-09-21T15:33:38.057489+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:33:38.057489+0000","last_undegraded":"2024-09-21T15:33:38.057489+0000","last_fullsized":"2024-09-21T15:33:38.057489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":31325499,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":520,"num_read_kb":18483,"num_write":109,"num_write_kb":13319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.777891+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:33:34.777891+0000","last_peered":"2024-09-21T15:33:34.777891+0000","last_clean":"2024-09-21T15:33:34.777891+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:33:34.777891+0000","last_undegraded":"2024-09-21T15:33:34.777891+0000","last_fullsized":"2024-09-21T15:33:34.777891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"63'94","reported_seq":372,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.967116+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:33:37.967116+0000","last_peered":"2024-09-21T15:33:37.967116+0000","last_clean":"2024-09-21T15:33:37.967116+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:33:37.967116+0000","last_undegraded":"2024-09-21T15:33:37.967116+0000","last_fullsized":"2024-09-21T15:33:37.967116+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":4956084,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":548,"num_read_kb":925,"num_write":100,"num_write_kb":971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.774577+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:33:34.774577+0000","last_peered":"2024-09-21T15:33:34.774577+0000","last_clean":"2024-09-21T15:33:34.774577+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:33:34.774577+0000","last_undegraded":"2024-09-21T15:33:34.774577+0000","last_fullsized":"2024-09-21T15:33:34.774577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"63'297","reported_seq":497,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.079153+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:33:38.079153+0000","last_peered":"2024-09-21T15:33:38.079153+0000","last_clean":"2024-09-21T15:33:38.079153+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:33:38.079153+0000","last_undegraded":"2024-09-21T15:33:38.079153+0000","last_fullsized":"2024-09-21T15:33:38.079153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":6792850,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":1120,"num_read_kb":1995,"num_write":388,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"63'83","reported_seq":322,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.056926+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:33:38.056926+0000","last_peered":"2024-09-21T15:33:38.056926+0000","last_clean":"2024-09-21T15:33:38.056926+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:33:38.056926+0000","last_undegraded":"2024-09-21T15:33:38.056926+0000","last_fullsized":"2024-09-21T15:33:38.056926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":10227975,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":209,"num_read_kb":1326,"num_write":68,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":60,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.779832+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:33:34.779832+0000","last_peered":"2024-09-21T15:33:34.779832+0000","last_clean":"2024-09-21T15:33:34.779832+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:33:34.779832+0000","last_undegraded":"2024-09-21T15:33:34.779832+0000","last_fullsized":"2024-09-21T15:33:34.779832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"63'334","reported_seq":832,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:36.779220+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:33:36.779220+0000","last_peered":"2024-09-21T15:33:36.779220+0000","last_clean":"2024-09-21T15:33:36.779220+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:33:36.779220+0000","last_undegraded":"2024-09-21T15:33:36.779220+0000","last_fullsized":"2024-09-21T15:33:36.779220+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":23016137,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":1807,"num_read_kb":1585,"num_write":441,"num_write_kb":1111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.932895+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:33:34.932895+0000","last_peered":"2024-09-21T15:33:34.932895+0000","last_clean":"2024-09-21T15:33:34.932895+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:33:34.932895+0000","last_undegraded":"2024-09-21T15:33:34.932895+0000","last_fullsized":"2024-09-21T15:33:34.932895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"63'160","reported_seq":480,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.746012+0000","last_change":"2024-09-21T15:33:29.748980+0000","last_active":"2024-09-21T15:33:37.746012+0000","last_peered":"2024-09-21T15:33:37.746012+0000","last_clean":"2024-09-21T15:33:37.746012+0000","last_became_active":"2024-09-21T15:33:29.747834+0000","last_became_peered":"2024-09-21T15:33:29.747834+0000","last_unstale":"2024-09-21T15:33:37.746012+0000","last_undegraded":"2024-09-21T15:33:37.746012+0000","last_fullsized":"2024-09-21T15:33:37.746012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:53.405024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":16081955,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":613,"num_read_kb":1053,"num_write":213,"num_write_kb":9772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":399716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":59,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.777805+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:33:34.777805+0000","last_peered":"2024-09-21T15:33:34.777805+0000","last_clean":"2024-09-21T15:33:34.777805+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:33:34.777805+0000","last_undegraded":"2024-09-21T15:33:34.777805+0000","last_fullsized":"2024-09-21T15:33:34.777805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'82","reported_seq":266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.208445+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:33:37.208445+0000","last_peered":"2024-09-21T15:33:37.208445+0000","last_clean":"2024-09-21T15:33:37.208445+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:33:37.208445+0000","last_undegraded":"2024-09-21T15:33:37.208445+0000","last_fullsized":"2024-09-21T15:33:37.208445+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":5013003,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":170,"num_read_kb":2147,"num_write":69,"num_write_kb":1183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":83,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.779878+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:33:34.779878+0000","last_peered":"2024-09-21T15:33:34.779878+0000","last_clean":"2024-09-21T15:33:34.779878+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:33:34.779878+0000","last_undegraded":"2024-09-21T15:33:34.779878+0000","last_fullsized":"2024-09-21T15:33:34.779878+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:34.771603+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:33:34.771603+0000","last_peered":"2024-09-21T15:33:34.771603+0000","last_clean":"2024-09-21T15:33:34.771603+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:33:34.771603+0000","last_undegraded":"2024-09-21T15:33:34.771603+0000","last_fullsized":"2024-09-21T15:33:34.771603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"63'372","reported_seq":913,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.440694+0000","last_change":"2024-09-21T15:33:29.749882+0000","last_active":"2024-09-21T15:33:37.440694+0000","last_peered":"2024-09-21T15:33:37.440694+0000","last_clean":"2024-09-21T15:33:37.440694+0000","last_became_active":"2024-09-21T15:33:29.748534+0000","last_became_peered":"2024-09-21T15:33:29.748534+0000","last_unstale":"2024-09-21T15:33:37.440694+0000","last_undegraded":"2024-09-21T15:33:37.440694+0000","last_fullsized":"2024-09-21T15:33:37.440694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:33.004724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":7390359,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":2333,"num_read_kb":3459,"num_write":516,"num_write_kb":2472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":786567,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"63'84","reported_seq":322,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.661743+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:33:37.661743+0000","last_peered":"2024-09-21T15:33:37.661743+0000","last_clean":"2024-09-21T15:33:37.661743+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:33:37.661743+0000","last_undegraded":"2024-09-21T15:33:37.661743+0000","last_fullsized":"2024-09-21T15:33:37.661743+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":20425761,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":204,"num_read_kb":2179,"num_write":88,"num_write_kb":1347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"63'142","reported_seq":526,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.029972+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:33:38.029972+0000","last_peered":"2024-09-21T15:33:38.029972+0000","last_clean":"2024-09-21T15:33:38.029972+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:33:38.029972+0000","last_undegraded":"2024-09-21T15:33:38.029972+0000","last_fullsized":"2024-09-21T15:33:38.029972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":11488780,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":629,"num_read_kb":4861,"num_write":136,"num_write_kb":2323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"62'172","reported_seq":980,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T15:33:35.231532+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:33:35.231532+0000","last_peered":"2024-09-21T15:33:35.231532+0000","last_clean":"2024-09-21T15:33:35.231532+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:33:35.231532+0000","last_undegraded":"2024-09-21T15:33:35.231532+0000","last_fullsized":"2024-09-21T15:33:35.231532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":26545203,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":1881,"num_read_kb":7505,"num_write":217,"num_write_kb":6854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"63'115","reported_seq":330,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:38.067826+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:33:38.067826+0000","last_peered":"2024-09-21T15:33:38.067826+0000","last_clean":"2024-09-21T15:33:38.067826+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:33:38.067826+0000","last_undegraded":"2024-09-21T15:33:38.067826+0000","last_fullsized":"2024-09-21T15:33:38.067826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":23866230,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":265,"num_read_kb":3193,"num_write":127,"num_write_kb":9395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"63'72","reported_seq":288,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.238138+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:33:37.238138+0000","last_peered":"2024-09-21T15:33:37.238138+0000","last_clean":"2024-09-21T15:33:37.238138+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:33:37.238138+0000","last_undegraded":"2024-09-21T15:33:37.238138+0000","last_fullsized":"2024-09-21T15:33:37.238138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":11706256,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":198,"num_read_kb":1189,"num_write":75,"num_write_kb":8309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"63'98","reported_seq":1337,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.615342+0000","last_change":"2024-09-21T15:33:29.818868+0000","last_active":"2024-09-21T15:33:37.615342+0000","last_peered":"2024-09-21T15:33:37.615342+0000","last_clean":"2024-09-21T15:33:37.615342+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:33:37.615342+0000","last_undegraded":"2024-09-21T15:33:37.615342+0000","last_fullsized":"2024-09-21T15:33:37.615342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:51:03.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":22306771,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1187,"num_read_kb":4085,"num_write":118,"num_write_kb":10715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'191","reported_seq":961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:36.782431+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:33:36.782431+0000","last_peered":"2024-09-21T15:33:36.782431+0000","last_clean":"2024-09-21T15:33:36.782431+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:33:36.782431+0000","last_undegraded":"2024-09-21T15:33:36.782431+0000","last_fullsized":"2024-09-21T15:33:36.782431+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":18385947,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":1863,"num_read_kb":3456,"num_write":218,"num_write_kb":1869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"61'72","reported_seq":330,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.027217+0000","last_change":"2024-09-21T15:32:48.755902+0000","last_active":"2024-09-21T15:33:37.027217+0000","last_peered":"2024-09-21T15:33:37.027217+0000","last_clean":"2024-09-21T15:33:37.027217+0000","last_became_active":"2024-09-21T15:32:46.733304+0000","last_became_peered":"2024-09-21T15:32:46.733304+0000","last_unstale":"2024-09-21T15:33:37.027217+0000","last_undegraded":"2024-09-21T15:33:37.027217+0000","last_fullsized":"2024-09-21T15:33:37.027217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:49:44.619685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":8373746,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":232,"num_read_kb":1428,"num_write":53,"num_write_kb":1023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"63'309","reported_seq":1113,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.024195+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:33:37.024195+0000","last_peered":"2024-09-21T15:33:37.024195+0000","last_clean":"2024-09-21T15:33:37.024195+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:33:37.024195+0000","last_undegraded":"2024-09-21T15:33:37.024195+0000","last_fullsized":"2024-09-21T15:33:37.024195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":20031535,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":2443,"num_read_kb":8175,"num_write":403,"num_write_kb":13093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'213","reported_seq":409,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:36.779756+0000","last_change":"2024-09-21T15:33:30.754648+0000","last_active":"2024-09-21T15:33:36.779756+0000","last_peered":"2024-09-21T15:33:36.779756+0000","last_clean":"2024-09-21T15:33:36.779756+0000","last_became_active":"2024-09-21T15:33:30.754007+0000","last_became_peered":"2024-09-21T15:33:30.754007+0000","last_unstale":"2024-09-21T15:33:36.779756+0000","last_undegraded":"2024-09-21T15:33:36.779756+0000","last_fullsized":"2024-09-21T15:33:36.779756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:12.343754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":5293202,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":683,"num_read_kb":1009,"num_write":277,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":59879,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"63'190","reported_seq":1432,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-21T15:33:37.746605+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:33:37.746605+0000","last_peered":"2024-09-21T15:33:37.746605+0000","last_clean":"2024-09-21T15:33:37.746605+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:33:37.746605+0000","last_undegraded":"2024-09-21T15:33:37.746605+0000","last_fullsized":"2024-09-21T15:33:37.746605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":16591707,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":3510,"num_read_kb":4077,"num_write":184,"num_write_kb":1732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":513389747,"num_objects":796,"num_object_clones":166,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":353,"num_read":38562,"num_read_kb":116840,"num_write":6772,"num_write_kb":162822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4607974,"num_keys_recovered":35,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":80945152,"data_stored":96146602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5573,"ondisk_log_size":5573,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53492,"kb_used_data":14188,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94318348,"statfs":{"total":96636764160,"available":96581988352,"internally_reserved":0,"allocated":14528512,"data_stored":16349200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.2}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.90300000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.02},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.86199999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.1679999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.94599999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.089},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.1319999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.0529999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42340,"kb_used_data":7132,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329500,"statfs":{"total":96636764160,"available":96593408000,"internally_reserved":0,"allocated":7303168,"data_stored":6939917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.67200000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.163}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.054},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.2869999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.405},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.86199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44236,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94327604,"statfs":{"total":96636764160,"available":96591466496,"internally_reserved":0,"allocated":9240576,"data_stored":11238497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.111},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0860000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.755}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.629},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.87}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.083}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0620000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50600,"kb_used_data":11296,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94321240,"statfs":{"total":96636764160,"available":96584949760,"internally_reserved":0,"allocated":11567104,"data_stored":11303868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":2.6739999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.755}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":3.069},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":2.952}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":3.1480000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":2.8769999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":3.218}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":3.0289999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":6.3369999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":4.5469999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":4.8019999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":2.9009999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":3.2839999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55724,"kb_used_data":20488,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94316116,"statfs":{"total":96636764160,"available":96579702784,"internally_reserved":0,"allocated":20979712,"data_stored":25164671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.28599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.34300000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.36699999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.503},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.63800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":2486272,"data_stored":2486944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.58999999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.60699999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92800000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.754}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.879}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43388,"kb_used_data":8180,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328452,"statfs":{"total":96636764160,"available":96592334848,"internally_reserved":0,"allocated":8376320,"data_stored":9509068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.47699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41499999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.374},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.45600000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.93600000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.871}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.98999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":11308,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":11579392,"data_stored":15122802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.625},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.76700000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.749},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.04}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.075}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.264},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.0269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10895360,"data_stored":14823316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7639040,"data_stored":9180418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1753088,"data_stored":2158313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20594688,"data_stored":25174191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11546624,"data_stored":11658594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8646656,"data_stored":11028282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5976064,"data_stored":6020918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13893632,"data_stored":16102570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:33:41.029 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:41.027+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:41.059 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.059 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:41.160 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:41.159+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:41.231 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:41.227+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:41.237 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.237 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.250 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:41.344 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:41.343+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:41.413 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.414 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:41.433 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:41.433+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:41.534 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:41.533+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:41.595 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.595 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:41.652 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:41.651+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:41.734 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.734 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.735 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:41.731+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:41.747 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:41.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:33:41.835 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:41.833+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:41.919 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:41.919 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:41.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:41.935 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:41.931+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:42.037 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:42.037+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:42.053 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.054 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.068 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:42.138 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:42.137+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:42.185 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.185 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:42.244 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:42.243+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:42.322 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.323 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:42.380 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:42.379+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:42.439 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:42.437+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:42.476 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.476 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.494 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:42.540 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:42.539+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:42.641 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:42.639+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:42.669 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.670 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:42.741 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:42.741+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:42.832 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:33:42.833 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:42.833 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:42.842 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:42.839+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:42.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:42.848 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:42.943 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:42.939+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:43.004 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:43.004 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:43.019 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:43.020 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:43.024 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:43.025 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:43.035 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:43.035 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:43.039 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:43.043 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:43.039+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:43.052 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:43.052 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:43.071 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:43.095 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:43.117 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:43.137 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:43.138 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:43.144 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:43.143+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:43.167 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:43.167 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:43.167 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:43.167 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:43.167 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:43.183 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:43.244 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:43.245+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:43.301 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.511748646000001, 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:43.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.307 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 64.505137439999999, 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:43.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:43.313 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 50.930309721, 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.314 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.628495727999997, 2024-09-21T15:33:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.324 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.549679073999997, 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:43.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:43.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.519326061000001, 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.328 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:43.333 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.511454366000002, 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:43.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:43.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:43.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:43.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.500274918000002, 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.339 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:43.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.340 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:43.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.345 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.495042552000001, 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:43.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.349 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.476557270999997, 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.353 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.349403768999998, 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.354 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:43.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:43.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:43.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:43.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.349247447000003, 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:43.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:43.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.343565222000002, 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:43.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:43.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:43.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.342602403000001, 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.370 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:43.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:43.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.58175748, 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.374 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 35.272732845999997, 2024-09-21T15:33:43.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.993667868999999, 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:43.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:43.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:43.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 26.662828372, 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.391 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:43.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26861 3.1c 3:3bbf239b:::rbd_data.1191e6bca1cd.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.992979938, 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0087661500000000003, 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26861 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.397 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306150+0000", 2024-09-21T15:33:43.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307085+0000", 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000935209 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307123+0000", 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8003999999999997e-05 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307141+0000", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7842e-05 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307265+0000", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124389 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307530+0000", 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026449399999999999 2024-09-21T15:33:43.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307916+0000", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038618499999999999 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314315+0000", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063993590000000003 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314679+0000", 2024-09-21T15:33:43.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036339 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314820+0000", 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014107400000000001 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314919+0000", 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9418000000000003e-05 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26864 3.1c 3:3c39757f:::rbd_data.1191e6bca1cd.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 2.9896860250000001, 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077293780000000003, 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26864 2024-09-21T15:33:43.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309444+0000", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310084+0000", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00063922000000000002 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310138+0000", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4676000000000003e-05 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310160+0000", 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1325999999999999e-05 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310956+0000", 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00079680599999999995 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311180+0000", 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337600000000001 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311734+0000", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055437499999999996 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.315365+0000", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036305460000000001 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.316842+0000", 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001476904 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317088+0000", 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024673799999999998 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317176+0000", 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8107000000000002e-05 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.408 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:43.410 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:43.410 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:43.407+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:43.510 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:43.507+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:43.580 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:43.580 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:43.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:43.640 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:43.641+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:43.711 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:43.707+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:43.712 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:43.712 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:43.712 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:43.712 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:43.712 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:43.726 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:43.812 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:43.811+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.896 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.431558793999997, 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:43.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.900 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.201281418000001, 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.901 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:43.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:43.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:43.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.899910626999997, 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.906 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:43.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:43.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.224396806999998, 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:43.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:43.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:43.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:43.916 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.895540445, 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:43.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:43.920 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8955239529999997, 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.921 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:43.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:43.924 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8955075810000004, 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:43.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:43.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:43.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.895490916, 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:43.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:43.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8954740940000008, 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.933 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:43.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8951461760000008, 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:43.937 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8951155340000003, 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:43.941 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8950973900000001, 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:43.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.949 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8950669340000008, 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:43.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:43.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:43.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8950486459999993, 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8950313580000007, 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.961 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8950146920000002, 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.962 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:43.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:43.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:43.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8949903450000001, 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8949742030000003, 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.970 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:43.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:43.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:43.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8949576579999992, 2024-09-21T15:33:43.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:43.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.978 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.8949301139999992, 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:43.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:43.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:43.982 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:43.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:43.984 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:43.981+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:44.084 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:44.083+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:44.160 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:44.160 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:44.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:44.185 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:44.183+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:44.286 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:44.285+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:44.292 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:44.292 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:44.292 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:44.292 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:44.292 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:44.306 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:44.392 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:44.393+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:44.473 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1278 3.d 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 50.688977266999999, 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.060972842999999999, 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.474 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1278 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784553+0000", 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.784551+0000", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801400+0000", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01684892 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801431+0000", 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1581999999999997e-05 2024-09-21T15:33:44.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801451+0000", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.948e-05 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801589+0000", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013792500000000001 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.801994+0000", 2024-09-21T15:33:44.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00040495700000000001 2024-09-21T15:33:44.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 5", 2024-09-21T15:33:44.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.802913+0000", 2024-09-21T15:33:44.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00091958600000000001 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.832473+0000", 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.029559932000000001 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845354+0000", 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012880796999999999 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845448+0000", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3746000000000007e-05 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845526+0000", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7972999999999994e-05 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:3973 3.e 3:706aad26:::rbd_data.119117b26f37.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11281964~20966] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:44.480 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.905384269999999, 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.059230786000000001, 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 3973 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568146+0000", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.568144+0000", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583870+0000", 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015726602999999999 2024-09-21T15:33:44.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583922+0000", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1239000000000002e-05 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.583942+0000", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0213000000000001e-05 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584121+0000", 2024-09-21T15:33:44.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000179469 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.584383+0000", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026151100000000002 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.585420+0000", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0010370189999999999 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.625790+0000", 2024-09-21T15:33:44.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040369751000000002 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627101+0000", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001311127 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627255+0000", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015414899999999999 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.627377+0000", 2024-09-21T15:33:44.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122009 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:4208 3.11 3:8be735f7:::rbd_data.119117b26f37.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.764049587999999, 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054071906000000003, 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.486 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 4208 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709481+0000", 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.709478+0000", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721564+0000", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012085933 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721605+0000", 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0862999999999998e-05 2024-09-21T15:33:44.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721625+0000", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9953e-05 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.721772+0000", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014629199999999999 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.722036+0000", 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000264415 2024-09-21T15:33:44.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.723537+0000", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0015015549999999999 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.762221+0000", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038683842000000003 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763323+0000", 2024-09-21T15:33:44.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011017150000000001 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763453+0000", 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130199 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.763553+0000", 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9382000000000005e-05 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.491 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:14128 3.11 3:8a0b4402:::rbd_data.1191bd191516.000000000000003e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 19.857914682000001, 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045981519999999998, 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 14128 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.492 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615616+0000", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615614+0000", 2024-09-21T15:33:44.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615629+0000", 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4788e-05 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615645+0000", 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6401000000000001e-05 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615654+0000", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6160000000000002e-06 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615738+0000", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4012999999999997e-05 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.615954+0000", 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021574100000000001 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.625676+0000", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097218289999999995 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661339+0000", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.035663551000000002 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661449+0000", 2024-09-21T15:33:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109675 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661536+0000", 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7069000000000003e-05 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:24.661597+0000", 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1199999999999997e-05 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37474 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8295424~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178850+0000", 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.294680448, 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041197145999999997, 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37474 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178850+0000", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178850+0000", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178849+0000", 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178852+0000", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178858+0000", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8279999999999999e-06 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178862+0000", 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.219873+0000", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041010586000000002 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.219956+0000", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3887999999999994e-05 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220047+0000", 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0542000000000001e-05 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.501 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37475 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8004608~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178866+0000", 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2946642749999999, 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041320211000000003, 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37475 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.502 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178866+0000", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178866+0000", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178866+0000", 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178868+0000", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4279999999999999e-06 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178874+0000", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6289999999999998e-06 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178878+0000", 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1609999999999996e-06 2024-09-21T15:33:44.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220074+0000", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041195797999999999 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220129+0000", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5475e-05 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220186+0000", 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7046000000000002e-05 2024-09-21T15:33:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37476 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8299520~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178883+0000", 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2946473009999999, 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041512444000000003, 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37476 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178883+0000", 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178883+0000", 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178883+0000", 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178885+0000", 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178890+0000", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4369999999999998e-06 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178895+0000", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8080000000000004e-06 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220224+0000", 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041328528000000003 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220306+0000", 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1815000000000002e-05 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220395+0000", 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9930000000000001e-05 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37477 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8008704~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.510 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178917+0000", 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2946135299999999, 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041616747000000003, 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37477 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178917+0000", 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178917+0000", 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178916+0000", 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178919+0000", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.1820000000000002e-06 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178929+0000", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9510000000000001e-06 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178941+0000", 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1292e-05 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220423+0000", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041482072000000002 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220477+0000", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3950999999999999e-05 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220534+0000", 2024-09-21T15:33:44.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6901999999999998e-05 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37478 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8303616~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178946+0000", 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.294584709, 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041799998999999997, 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.515 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37478 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178946+0000", 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178946+0000", 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178945+0000", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178948+0000", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7030000000000002e-06 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178953+0000", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5650000000000001e-06 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178958+0000", 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220570+0000", 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041612326999999998 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220654+0000", 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3549999999999998e-05 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220746+0000", 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1946000000000004e-05 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37479 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8012800~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.178962+0000", 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2945684719999999, 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041920151000000003, 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37479 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178962+0000", 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178962+0000", 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178962+0000", 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178964+0000", 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1449999999999998e-06 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178970+0000", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4030000000000004e-06 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.178975+0000", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5009999999999998e-06 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220773+0000", 2024-09-21T15:33:44.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041798011000000003 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220827+0000", 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3999000000000003e-05 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.220882+0000", 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5386000000000001e-05 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37481 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8016896~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180112+0000", 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2934180989999999, 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041123844, 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37481 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180112+0000", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180112+0000", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180112+0000", 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180115+0000", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2610000000000002e-06 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180146+0000", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0895999999999997e-05 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180159+0000", 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699e-05 2024-09-21T15:33:44.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221126+0000", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040966710000000003 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221180+0000", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3773000000000002e-05 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221236+0000", 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6097999999999997e-05 2024-09-21T15:33:44.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37482 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8311808~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180169+0000", 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.293361279, 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041282632999999999, 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37482 2024-09-21T15:33:44.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180169+0000", 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180169+0000", 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180168+0000", 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180174+0000", 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.541e-06 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180183+0000", 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3130000000000004e-06 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180188+0000", 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4559999999999999e-06 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221273+0000", 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041084543000000001 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221358+0000", 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5419000000000003e-05 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221452+0000", 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3233999999999995e-05 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.532 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37483 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8020992~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180194+0000", 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2933364940000001, 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041397279000000002, 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37483 2024-09-21T15:33:44.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180194+0000", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180194+0000", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180193+0000", 2024-09-21T15:33:44.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180196+0000", 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8629999999999999e-06 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180203+0000", 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7299999999999999e-06 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180208+0000", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5739999999999997e-06 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221481+0000", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041273099000000001 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221534+0000", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3674000000000001e-05 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221591+0000", 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6759000000000003e-05 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37484 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8315904~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180212+0000", 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2933180100000001, 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041589056999999999, 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37484 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180212+0000", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180212+0000", 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180212+0000", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180215+0000", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8270000000000002e-06 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180221+0000", 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180227+0000", 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1850000000000003e-06 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221627+0000", 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041400507000000003 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221711+0000", 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3605999999999999e-05 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221801+0000", 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0605000000000004e-05 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37485 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8025088~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180231+0000", 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2932992249999999, 2024-09-21T15:33:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041786246999999999, 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37485 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180231+0000", 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180231+0000", 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180231+0000", 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180242+0000", 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1024e-05 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180249+0000", 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1899999999999998e-06 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180255+0000", 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.251e-06 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221829+0000", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041574247000000002 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.221901+0000", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1922999999999996e-05 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222017+0000", 2024-09-21T15:33:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011599600000000001 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37486 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180260+0000", 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2932707059999999, 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041801301999999999, 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:33:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37486 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180260+0000", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180260+0000", 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180259+0000", 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180262+0000", 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4940000000000002e-06 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180268+0000", 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.19e-06 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180282+0000", 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4146e-05 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222051+0000", 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041769207000000003 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222061+0000", 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6339999999999996e-06 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37487 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180287+0000", 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2932435660000001, 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041804147999999999, 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:33:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37487 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180287+0000", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180287+0000", 2024-09-21T15:33:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180286+0000", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180289+0000", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5809999999999999e-06 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180296+0000", 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9619999999999996e-06 2024-09-21T15:33:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180300+0000", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4270000000000001e-06 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222086+0000", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041785295 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222091+0000", 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2469999999999999e-06 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37488 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180306+0000", 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2932241630000001, 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041828381999999997, 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37488 2024-09-21T15:33:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180306+0000", 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180306+0000", 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180306+0000", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180308+0000", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.368e-06 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180314+0000", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1290000000000001e-06 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180319+0000", 2024-09-21T15:33:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7430000000000002e-06 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222125+0000", 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041806246999999998 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222135+0000", 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2140000000000002e-06 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37489 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180414+0000", 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2931159219999999, 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041735622, 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37489 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180414+0000", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180414+0000", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180414+0000", 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180418+0000", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9349999999999996e-06 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180426+0000", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6729999999999998e-06 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180432+0000", 2024-09-21T15:33:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8579999999999996e-06 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222146+0000", 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041713592000000001 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222150+0000", 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3329999999999998e-06 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:37490 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:43.180437+0000", 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 1.2930932500000001, 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041751009999999998, 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 37490 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180437+0000", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180437+0000", 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180437+0000", 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180439+0000", 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5739999999999998e-06 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180445+0000", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9680000000000003e-06 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.180450+0000", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5000000000000001e-06 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222179+0000", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.041729608000000001 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:43.222188+0000", 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7350000000000002e-06 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:44.566 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:44.567 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:44.565+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:44.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:44.669 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:44.669+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:44.728 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:44.728 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:44.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:44.770 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:44.769+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:44.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:33:44.868 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:44.868 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:44.868 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:44.868 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:44.868 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:44.870 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:44.869+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:44.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:44.972 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:44.971+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:45.049 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.338902304999998, 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.050 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:45.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:45.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.328580010000003, 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.059 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 43.401082785, 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:45.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:45.065 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 43.367171351000003, 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10948 3.1 3:82fda950:::rbd_data.11917b2c111e.0000000000000011:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:45.069 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.367208621000003, 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025101894999999999, 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10948 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681359+0000", 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681370+0000", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.135e-05 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681391+0000", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0633e-05 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681400+0000", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8109999999999992e-06 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681492+0000", 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2101000000000005e-05 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681709+0000", 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021704800000000001 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.687063+0000", 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053539759999999999 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706359+0000", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019296002 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706374+0000", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5245999999999999e-05 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706424+0000", 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9996e-05 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706463+0000", 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9069999999999997e-05 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11973 3.18 3:1bdac4bd:::rbd_data.11917b2c111e.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.947241348999999, 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019588250000000001, 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11973 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101326+0000", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101338+0000", 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2381000000000001e-05 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101361+0000", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2884e-05 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101372+0000", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0716999999999999e-05 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101427+0000", 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4626000000000002e-05 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101493+0000", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6115e-05 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.104640+0000", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003147501 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120784+0000", 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016143372999999999 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120799+0000", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5832999999999999e-05 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120871+0000", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1977000000000006e-05 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120917+0000", 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5269999999999999e-05 2024-09-21T15:33:45.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16568 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059991114000001, 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016067491999999999, 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16568 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988578+0000", 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988580+0000", 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.432e-06 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988584+0000", 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.765e-06 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988586+0000", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.599e-06 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004385+0000", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015799359999999998 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004428+0000", 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2552000000000002e-05 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004646+0000", 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021814800000000001 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16569 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059970823, 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016416218, 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16569 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988600+0000", 2024-09-21T15:33:45.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3519999999999999e-06 2024-09-21T15:33:45.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988603+0000", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4070000000000002e-06 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988606+0000", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7350000000000001e-06 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004678+0000", 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016071621000000001 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004772+0000", 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4575999999999997e-05 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005015+0000", 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000242752 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.094 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16570 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059961392, 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016891402999999999, 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16570 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988609+0000", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3659999999999999e-06 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988612+0000", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988615+0000", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5160000000000001e-06 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005071+0000", 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016456487999999998 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005205+0000", 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133638 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005500+0000", 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029470199999999998 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16571 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059952824, 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017340120000000001, 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16571 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988618+0000", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2559999999999999e-06 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988622+0000", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.087e-06 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988626+0000", 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1559999999999997e-06 2024-09-21T15:33:45.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005524+0000", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016897760000000001 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005594+0000", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9998000000000003e-05 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005957+0000", 2024-09-21T15:33:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036308 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16572 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059939930000001, 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017681184999999999, 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16572 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988632+0000", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.227e-06 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988638+0000", 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1990000000000003e-06 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988641+0000", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.835e-06 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005986+0000", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017345098999999999 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006062+0000", 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5685999999999993e-05 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006311+0000", 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000249452 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16574 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059926702, 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956159999999999, 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16574 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988644+0000", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.35e-06 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988647+0000", 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1750000000000001e-06 2024-09-21T15:33:45.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988650+0000", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.553e-06 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006336+0000", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017685671 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006376+0000", 2024-09-21T15:33:45.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0154999999999998e-05 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006599+0000", 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022348800000000001 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16575 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.05991642, 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01820097, 2024-09-21T15:33:45.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16575 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988654+0000", 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1930000000000001e-06 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988657+0000", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.016e-06 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988661+0000", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2200000000000001e-06 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006625+0000", 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017964285 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006669+0000", 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4363000000000002e-05 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006854+0000", 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000185105 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16723 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.115 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059542285999999, 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016156074999999999, 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16723 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:45.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989029+0000", 2024-09-21T15:33:45.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6929999999999999e-06 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989033+0000", 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3479999999999997e-06 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989036+0000", 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.565e-06 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005031+0000", 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015995019999999999 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005095+0000", 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4395999999999998e-05 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005184+0000", 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8259999999999999e-05 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16726 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059531938999999, 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016361140999999999, 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16726 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989039+0000", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.114e-06 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989062+0000", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2787000000000001e-05 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989066+0000", 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:33:45.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005228+0000", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016161933999999999 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005298+0000", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0092999999999995e-05 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005399+0000", 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010131700000000001 2024-09-21T15:33:45.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16729 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.05950123, 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016513967000000001, 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16729 2024-09-21T15:33:45.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989068+0000", 2024-09-21T15:33:45.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989070+0000", 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.809e-06 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989076+0000", 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0530000000000003e-06 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989083+0000", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9450000000000004e-06 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005427+0000", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016344145000000001 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005495+0000", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8040999999999994e-05 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005583+0000", 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7250000000000007e-05 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16732 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059483503999999, 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016636597999999999, 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.128 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16732 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989091+0000", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4739999999999996e-06 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989097+0000", 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9909999999999999e-06 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989102+0000", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.792e-06 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005607+0000", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016505407 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005670+0000", 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2288999999999999e-05 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005723+0000", 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3154e-05 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16735 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059463815999999, 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016763145, 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.132 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16735 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:45.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989105+0000", 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989108+0000", 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7149999999999998e-06 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989123+0000", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5291999999999999e-05 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989128+0000", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005749+0000", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016620945000000002 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005806+0000", 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6436e-05 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005869+0000", 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3303e-05 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16738 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:45.136 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059436910000001, 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016893127000000001, 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16738 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989132+0000", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989135+0000", 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4530000000000001e-06 2024-09-21T15:33:45.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989141+0000", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0599999999999996e-06 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989146+0000", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005907+0000", 2024-09-21T15:33:45.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016761426999999999 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005977+0000", 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0178000000000005e-05 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006026+0000", 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8862000000000001e-05 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16740 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.059420257999999, 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016975200999999999, 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16740 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989149+0000", 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989151+0000", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2170000000000001e-06 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989157+0000", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.631e-06 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989161+0000", 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4700000000000004e-06 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006040+0000", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016878955000000001 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006077+0000", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6838999999999997e-05 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006125+0000", 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7547999999999999e-05 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.144 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:45.145 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:45.145 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:45.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:45.146 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:45.143+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:45.159 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:45.248 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:45.243+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:45.313 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:45.313 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:45.313 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:45.314 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:45.329 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:45.340 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:45.340 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:45.348 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:45.345+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:45.357 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:45.358 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:45.376 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:45.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:45.412 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:45.413 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:45.432 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:45.449 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:45.449+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:45.452 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:45.452 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:45.454 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:45.455 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:45.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:45.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:45.472 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:45.550 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:45.549+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:45.585 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:45.585 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:45.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:45.651 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:45.649+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:45.730 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:45.731 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:45.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:45.752 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:45.749+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:45.853 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:45.853+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:45.858 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:33:45.875 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:45.876 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:45.891 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:45.981 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:45.981 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:45.982 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:33:45.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:33:45.984 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:45.983+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:46.055 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:46.051+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:46.069 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:46.069 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:46.084 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:46.172 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:46.171+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:46.243 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:46.243 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:46.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:46.257 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:46.255+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 67, 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:33:46.292 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:33:45.845560+0000", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:33:20.030364+0000", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5801984, 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "noscrub", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:33:46.293 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "jewel", 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:33:46.294 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:33:46.295 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:46.296 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:46.297 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:46.298 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.299 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:46.301 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:46.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:46.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:46.304 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:46.305 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:33:46.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:33:46.307 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "64", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:46.308 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 25, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 64, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:46.309 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:46.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:33:46.311 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:46.312 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.314 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:33:46.316 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:33:46.317 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 56, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.318 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.319 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:33:46.322 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.323 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.325 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.326 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.328 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.75054931640625, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 57, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.329 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.330 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:33:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:33:46.332 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.333 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.8086395263671875, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.334 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.335 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.336 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.337 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:33:46.338 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.339 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.344 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 53, 2024-09-21T15:33:46.345 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.346 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.348 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.349 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:33:46.350 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.71441650390625, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 56, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.353 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:33:46.355 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.358 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.359 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:46.361 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:33:46.362 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [], 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:33:46.363 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:46.364 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:46.366 INFO:tasks.thrashosds.thrasher:j is {'epoch': 67, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:33:45.845560+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:33:20.030364+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 0.75054931640625, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 0.8086395263671875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 0.71441650390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:33:46.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:33:46.367 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:46.363+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:46.466 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:46.463+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:46.487 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:46.488 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:46.503 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:46.567 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:46.565+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:46.668 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:46.665+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:46.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:46.726 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:46.727 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:33:46.742 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-21T15:33:45.854864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524331513,"num_objects":899,"num_object_clones":190,"num_object_copies":1798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":417,"num_read":58931,"num_read_kb":135912,"num_write":7135,"num_write_kb":172365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":5198342,"num_keys_recovered":35,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5918,"ondisk_log_size":5918,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386132,"kb_used_data":100328,"kb_used_omap":12,"kb_used_meta":285683,"kb_avail":754588588,"statfs":{"total":773094113280,"available":772698714112,"internally_reserved":0,"allocated":102735872,"data_stored":114496064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":292539961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10351379,"num_objects":99,"num_object_clones":24,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":64,"num_read":20253,"num_read_kb":18972,"num_write":202,"num_write_kb":7125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.199237"},"pg_stats":[{"pgid":"3.1f","version":"65'67","reported_seq":205,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.648144+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:33:40.648144+0000","last_peered":"2024-09-21T15:33:40.648144+0000","last_clean":"2024-09-21T15:33:40.648144+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:33:40.648144+0000","last_undegraded":"2024-09-21T15:33:40.648144+0000","last_fullsized":"2024-09-21T15:33:40.648144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":16927096,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":125,"num_read_kb":1734,"num_write":63,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"64'210","reported_seq":545,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.181107+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:33:43.181107+0000","last_peered":"2024-09-21T15:33:43.181107+0000","last_clean":"2024-09-21T15:33:43.181107+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:33:43.181107+0000","last_undegraded":"2024-09-21T15:33:43.181107+0000","last_fullsized":"2024-09-21T15:33:43.181107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":17516087,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":948,"num_read_kb":8003,"num_write":331,"num_write_kb":22553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"64'70","reported_seq":215,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872137+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:33:39.872137+0000","last_peered":"2024-09-21T15:33:39.872137+0000","last_clean":"2024-09-21T15:33:39.872137+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:33:39.872137+0000","last_undegraded":"2024-09-21T15:33:39.872137+0000","last_fullsized":"2024-09-21T15:33:39.872137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":12750223,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":129,"num_read_kb":979,"num_write":82,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"65'125","reported_seq":922,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.181491+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:33:43.181491+0000","last_peered":"2024-09-21T15:33:43.181491+0000","last_clean":"2024-09-21T15:33:43.181491+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:33:43.181491+0000","last_undegraded":"2024-09-21T15:33:43.181491+0000","last_fullsized":"2024-09-21T15:33:43.181491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":42955154,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":875,"num_read_kb":1402,"num_write":121,"num_write_kb":2405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"64'199","reported_seq":779,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809730+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:33:39.809730+0000","last_peered":"2024-09-21T15:33:39.809730+0000","last_clean":"2024-09-21T15:33:39.809730+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:33:39.809730+0000","last_undegraded":"2024-09-21T15:33:39.809730+0000","last_fullsized":"2024-09-21T15:33:39.809730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":8479084,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":1273,"num_read_kb":3203,"num_write":200,"num_write_kb":1778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"64'99","reported_seq":1317,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.805762+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:33:39.805762+0000","last_peered":"2024-09-21T15:33:39.805762+0000","last_clean":"2024-09-21T15:33:39.805762+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:33:39.805762+0000","last_undegraded":"2024-09-21T15:33:39.805762+0000","last_fullsized":"2024-09-21T15:33:39.805762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":7631140,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":1214,"num_read_kb":4452,"num_write":87,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"65'400","reported_seq":12510,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.755740+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:33:43.755740+0000","last_peered":"2024-09-21T15:33:43.755740+0000","last_clean":"2024-09-21T15:33:43.755740+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:33:43.755740+0000","last_undegraded":"2024-09-21T15:33:43.755740+0000","last_fullsized":"2024-09-21T15:33:43.755740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":39533085,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":13653,"num_read_kb":10542,"num_write":581,"num_write_kb":3019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'323","reported_seq":645,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809273+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:33:39.809273+0000","last_peered":"2024-09-21T15:33:39.809273+0000","last_clean":"2024-09-21T15:33:39.809273+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:33:39.809273+0000","last_undegraded":"2024-09-21T15:33:39.809273+0000","last_fullsized":"2024-09-21T15:33:39.809273+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":6918971,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1856,"num_read_kb":1808,"num_write":483,"num_write_kb":9868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"65'126","reported_seq":846,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:41.157486+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:33:41.157486+0000","last_peered":"2024-09-21T15:33:41.157486+0000","last_clean":"2024-09-21T15:33:41.157486+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:33:41.157486+0000","last_undegraded":"2024-09-21T15:33:41.157486+0000","last_fullsized":"2024-09-21T15:33:41.157486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":22553743,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":966,"num_read_kb":5964,"num_write":181,"num_write_kb":3984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"63'230","reported_seq":1334,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.759271+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:33:40.759271+0000","last_peered":"2024-09-21T15:33:40.759271+0000","last_clean":"2024-09-21T15:33:40.759271+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:33:40.759271+0000","last_undegraded":"2024-09-21T15:33:40.759271+0000","last_fullsized":"2024-09-21T15:33:40.759271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":23178028,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":2577,"num_read_kb":4116,"num_write":222,"num_write_kb":1825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"64'142","reported_seq":337,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.806033+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:33:39.806033+0000","last_peered":"2024-09-21T15:33:39.806033+0000","last_clean":"2024-09-21T15:33:39.806033+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:33:39.806033+0000","last_undegraded":"2024-09-21T15:33:39.806033+0000","last_fullsized":"2024-09-21T15:33:39.806033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":4346145,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":425,"num_read_kb":4035,"num_write":173,"num_write_kb":2227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"64'332","reported_seq":1231,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.806105+0000","last_change":"2024-09-21T15:32:48.726261+0000","last_active":"2024-09-21T15:33:39.806105+0000","last_peered":"2024-09-21T15:33:39.806105+0000","last_clean":"2024-09-21T15:33:39.806105+0000","last_became_active":"2024-09-21T15:32:46.731389+0000","last_became_peered":"2024-09-21T15:32:46.731389+0000","last_unstale":"2024-09-21T15:33:39.806105+0000","last_undegraded":"2024-09-21T15:33:39.806105+0000","last_fullsized":"2024-09-21T15:33:39.806105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:02.678176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":11996821,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":2843,"num_read_kb":3120,"num_write":399,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.528313+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:33:40.528313+0000","last_peered":"2024-09-21T15:33:40.528313+0000","last_clean":"2024-09-21T15:33:40.528313+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:33:40.528313+0000","last_undegraded":"2024-09-21T15:33:40.528313+0000","last_fullsized":"2024-09-21T15:33:40.528313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"64'175","reported_seq":1216,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809770+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:33:39.809770+0000","last_peered":"2024-09-21T15:33:39.809770+0000","last_clean":"2024-09-21T15:33:39.809770+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:33:39.809770+0000","last_undegraded":"2024-09-21T15:33:39.809770+0000","last_fullsized":"2024-09-21T15:33:39.809770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":11472147,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":2570,"num_read_kb":4860,"num_write":159,"num_write_kb":1720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.807070+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:33:39.807070+0000","last_peered":"2024-09-21T15:33:39.807070+0000","last_clean":"2024-09-21T15:33:39.807070+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:33:39.807070+0000","last_undegraded":"2024-09-21T15:33:39.807070+0000","last_fullsized":"2024-09-21T15:33:39.807070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"64'95","reported_seq":664,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.757703+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:33:40.757703+0000","last_peered":"2024-09-21T15:33:40.757703+0000","last_clean":"2024-09-21T15:33:40.757703+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:33:40.757703+0000","last_undegraded":"2024-09-21T15:33:40.757703+0000","last_fullsized":"2024-09-21T15:33:40.757703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":31325499,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":562,"num_read_kb":18524,"num_write":112,"num_write_kb":13319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.315179+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:33:40.315179+0000","last_peered":"2024-09-21T15:33:40.315179+0000","last_clean":"2024-09-21T15:33:40.315179+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:33:40.315179+0000","last_undegraded":"2024-09-21T15:33:40.315179+0000","last_fullsized":"2024-09-21T15:33:40.315179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"65'97","reported_seq":421,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.724907+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:33:40.724907+0000","last_peered":"2024-09-21T15:33:40.724907+0000","last_clean":"2024-09-21T15:33:40.724907+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:33:40.724907+0000","last_undegraded":"2024-09-21T15:33:40.724907+0000","last_fullsized":"2024-09-21T15:33:40.724907+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":5480372,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":593,"num_read_kb":1269,"num_write":103,"num_write_kb":1483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.806842+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:33:39.806842+0000","last_peered":"2024-09-21T15:33:39.806842+0000","last_clean":"2024-09-21T15:33:39.806842+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:33:39.806842+0000","last_undegraded":"2024-09-21T15:33:39.806842+0000","last_fullsized":"2024-09-21T15:33:39.806842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"64'311","reported_seq":569,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872243+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:33:39.872243+0000","last_peered":"2024-09-21T15:33:39.872243+0000","last_clean":"2024-09-21T15:33:39.872243+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:33:39.872243+0000","last_undegraded":"2024-09-21T15:33:39.872243+0000","last_fullsized":"2024-09-21T15:33:39.872243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":7120530,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":1303,"num_read_kb":2143,"num_write":399,"num_write_kb":1739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"64'92","reported_seq":335,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.805973+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:33:39.805973+0000","last_peered":"2024-09-21T15:33:39.805973+0000","last_clean":"2024-09-21T15:33:39.805973+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:33:39.805973+0000","last_undegraded":"2024-09-21T15:33:39.805973+0000","last_fullsized":"2024-09-21T15:33:39.805973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":10454531,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":218,"num_read_kb":1329,"num_write":74,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":63,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872156+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:33:39.872156+0000","last_peered":"2024-09-21T15:33:39.872156+0000","last_clean":"2024-09-21T15:33:39.872156+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:33:39.872156+0000","last_undegraded":"2024-09-21T15:33:39.872156+0000","last_fullsized":"2024-09-21T15:33:39.872156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"66'351","reported_seq":885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.068377+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:33:43.068377+0000","last_peered":"2024-09-21T15:33:43.068377+0000","last_clean":"2024-09-21T15:33:43.068377+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:33:43.068377+0000","last_undegraded":"2024-09-21T15:33:43.068377+0000","last_fullsized":"2024-09-21T15:33:43.068377+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":23272137,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":1907,"num_read_kb":1667,"num_write":451,"num_write_kb":1118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.094583+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:33:40.094583+0000","last_peered":"2024-09-21T15:33:40.094583+0000","last_clean":"2024-09-21T15:33:40.094583+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:33:40.094583+0000","last_undegraded":"2024-09-21T15:33:40.094583+0000","last_fullsized":"2024-09-21T15:33:40.094583+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"64'163","reported_seq":485,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809472+0000","last_change":"2024-09-21T15:33:29.748980+0000","last_active":"2024-09-21T15:33:39.809472+0000","last_peered":"2024-09-21T15:33:39.809472+0000","last_clean":"2024-09-21T15:33:39.809472+0000","last_became_active":"2024-09-21T15:33:29.747834+0000","last_became_peered":"2024-09-21T15:33:29.747834+0000","last_unstale":"2024-09-21T15:33:39.809472+0000","last_undegraded":"2024-09-21T15:33:39.809472+0000","last_fullsized":"2024-09-21T15:33:39.809472+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:53.405024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":16196478,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":614,"num_read_kb":1053,"num_write":215,"num_write_kb":9772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":399716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.314053+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:33:40.314053+0000","last_peered":"2024-09-21T15:33:40.314053+0000","last_clean":"2024-09-21T15:33:40.314053+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:33:40.314053+0000","last_undegraded":"2024-09-21T15:33:40.314053+0000","last_fullsized":"2024-09-21T15:33:40.314053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"64'91","reported_seq":278,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872029+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:33:39.872029+0000","last_peered":"2024-09-21T15:33:39.872029+0000","last_clean":"2024-09-21T15:33:39.872029+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:33:39.872029+0000","last_undegraded":"2024-09-21T15:33:39.872029+0000","last_fullsized":"2024-09-21T15:33:39.872029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":5144075,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":20,"num_read":175,"num_read_kb":2147,"num_write":74,"num_write_kb":1183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":86,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872058+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:33:39.872058+0000","last_peered":"2024-09-21T15:33:39.872058+0000","last_clean":"2024-09-21T15:33:39.872058+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:33:39.872058+0000","last_undegraded":"2024-09-21T15:33:39.872058+0000","last_fullsized":"2024-09-21T15:33:39.872058+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809412+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:33:39.809412+0000","last_peered":"2024-09-21T15:33:39.809412+0000","last_clean":"2024-09-21T15:33:39.809412+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:33:39.809412+0000","last_undegraded":"2024-09-21T15:33:39.809412+0000","last_fullsized":"2024-09-21T15:33:39.809412+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"65'375","reported_seq":934,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.445737+0000","last_change":"2024-09-21T15:33:29.749882+0000","last_active":"2024-09-21T15:33:40.445737+0000","last_peered":"2024-09-21T15:33:40.445737+0000","last_clean":"2024-09-21T15:33:40.445737+0000","last_became_active":"2024-09-21T15:33:29.748534+0000","last_became_peered":"2024-09-21T15:33:29.748534+0000","last_unstale":"2024-09-21T15:33:40.445737+0000","last_undegraded":"2024-09-21T15:33:40.445737+0000","last_fullsized":"2024-09-21T15:33:40.445737+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:33.004724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":7914647,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":2351,"num_read_kb":3499,"num_write":519,"num_write_kb":2984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":786567,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"65'91","reported_seq":363,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.180817+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:33:43.180817+0000","last_peered":"2024-09-21T15:33:43.180817+0000","last_clean":"2024-09-21T15:33:43.180817+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:33:43.180817+0000","last_undegraded":"2024-09-21T15:33:43.180817+0000","last_fullsized":"2024-09-21T15:33:43.180817+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":21779598,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":14,"num_read":239,"num_read_kb":2353,"num_write":95,"num_write_kb":2382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"64'155","reported_seq":575,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.757386+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:33:40.757386+0000","last_peered":"2024-09-21T15:33:40.757386+0000","last_clean":"2024-09-21T15:33:40.757386+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:33:40.757386+0000","last_undegraded":"2024-09-21T15:33:40.757386+0000","last_fullsized":"2024-09-21T15:33:40.757386+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":11488780,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":670,"num_read_kb":4903,"num_write":143,"num_write_kb":2324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"65'175","reported_seq":986,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:41.156304+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:33:41.156304+0000","last_peered":"2024-09-21T15:33:41.156304+0000","last_clean":"2024-09-21T15:33:41.156304+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:33:41.156304+0000","last_undegraded":"2024-09-21T15:33:41.156304+0000","last_fullsized":"2024-09-21T15:33:41.156304+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":27069491,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":1883,"num_read_kb":7505,"num_write":220,"num_write_kb":7366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'123","reported_seq":339,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.872300+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:33:39.872300+0000","last_peered":"2024-09-21T15:33:39.872300+0000","last_clean":"2024-09-21T15:33:39.872300+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:33:39.872300+0000","last_undegraded":"2024-09-21T15:33:39.872300+0000","last_fullsized":"2024-09-21T15:33:39.872300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":23866230,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":267,"num_read_kb":3193,"num_write":131,"num_write_kb":9395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"65'78","reported_seq":297,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:41.478608+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:33:41.478608+0000","last_peered":"2024-09-21T15:33:41.478608+0000","last_clean":"2024-09-21T15:33:41.478608+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:33:41.478608+0000","last_undegraded":"2024-09-21T15:33:41.478608+0000","last_fullsized":"2024-09-21T15:33:41.478608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":12754832,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":202,"num_read_kb":1189,"num_write":82,"num_write_kb":9237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"64'103","reported_seq":9679,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-21T15:33:43.242363+0000","last_change":"2024-09-21T15:33:29.818868+0000","last_active":"2024-09-21T15:33:43.242363+0000","last_peered":"2024-09-21T15:33:43.242363+0000","last_clean":"2024-09-21T15:33:43.242363+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:33:43.242363+0000","last_undegraded":"2024-09-21T15:33:43.242363+0000","last_fullsized":"2024-09-21T15:33:43.242363+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:51:03.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":22779782,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":9525,"num_read_kb":12586,"num_write":121,"num_write_kb":10722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"64'194","reported_seq":967,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.806169+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:33:39.806169+0000","last_peered":"2024-09-21T15:33:39.806169+0000","last_clean":"2024-09-21T15:33:39.806169+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:33:39.806169+0000","last_undegraded":"2024-09-21T15:33:39.806169+0000","last_fullsized":"2024-09-21T15:33:39.806169+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":18517019,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":1865,"num_read_kb":3456,"num_write":220,"num_write_kb":1869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"64'77","reported_seq":342,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.528293+0000","last_change":"2024-09-21T15:32:48.755902+0000","last_active":"2024-09-21T15:33:40.528293+0000","last_peered":"2024-09-21T15:33:40.528293+0000","last_clean":"2024-09-21T15:33:40.528293+0000","last_became_active":"2024-09-21T15:32:46.733304+0000","last_became_peered":"2024-09-21T15:32:46.733304+0000","last_unstale":"2024-09-21T15:33:40.528293+0000","last_undegraded":"2024-09-21T15:33:40.528293+0000","last_fullsized":"2024-09-21T15:33:40.528293+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:49:44.619685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":8373762,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":20,"num_read":242,"num_read_kb":1430,"num_write":56,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"65'315","reported_seq":1152,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:41.664322+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:33:41.664322+0000","last_peered":"2024-09-21T15:33:41.664322+0000","last_clean":"2024-09-21T15:33:41.664322+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:33:41.664322+0000","last_undegraded":"2024-09-21T15:33:41.664322+0000","last_fullsized":"2024-09-21T15:33:41.664322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":21733050,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":2549,"num_read_kb":8262,"num_write":410,"num_write_kb":14629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"64'217","reported_seq":415,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:39.809444+0000","last_change":"2024-09-21T15:33:30.754648+0000","last_active":"2024-09-21T15:33:39.809444+0000","last_peered":"2024-09-21T15:33:39.809444+0000","last_clean":"2024-09-21T15:33:39.809444+0000","last_became_active":"2024-09-21T15:33:30.754007+0000","last_became_peered":"2024-09-21T15:33:30.754007+0000","last_unstale":"2024-09-21T15:33:39.809444+0000","last_undegraded":"2024-09-21T15:33:39.809444+0000","last_fullsized":"2024-09-21T15:33:39.809444+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:12.343754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":5293202,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":684,"num_read_kb":1009,"num_write":279,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":59879,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"64'197","reported_seq":1441,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-21T15:33:40.314549+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:33:40.314549+0000","last_peered":"2024-09-21T15:33:40.314549+0000","last_clean":"2024-09-21T15:33:40.314549+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:33:40.314549+0000","last_undegraded":"2024-09-21T15:33:40.314549+0000","last_fullsized":"2024-09-21T15:33:40.314549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":16919387,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":3512,"num_read_kb":4077,"num_write":188,"num_write_kb":1742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":523741126,"num_objects":895,"num_object_clones":190,"num_object_copies":1790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":895,"num_whiteouts":417,"num_read":58815,"num_read_kb":135812,"num_write":6974,"num_write_kb":169947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4607974,"num_keys_recovered":35,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":95404032,"data_stored":110555828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5798,"ondisk_log_size":5798,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56628,"kb_used_data":17320,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94315212,"statfs":{"total":96636764160,"available":96578777088,"internally_reserved":0,"allocated":17735680,"data_stored":19512908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.518},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.26000000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.534},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.5}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.478}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.4990000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42388,"kb_used_data":7172,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329452,"statfs":{"total":96636764160,"available":96593358848,"internally_reserved":0,"allocated":7344128,"data_stored":6957686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.77100000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.626},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.69099999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.95299999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.98199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.82099999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.86799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47328,"kb_used_data":12116,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94324512,"statfs":{"total":96636764160,"available":96588300288,"internally_reserved":0,"allocated":12406784,"data_stored":14357738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.59299999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76700000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0509999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.081},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.171}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.028}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54936,"kb_used_data":15628,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94316904,"statfs":{"total":96636764160,"available":96580509696,"internally_reserved":0,"allocated":16003072,"data_stored":15689903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.59699999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85699999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.93999999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1020000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.90000000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0720000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.181},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.381}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56628,"kb_used_data":21412,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94315212,"statfs":{"total":96636764160,"available":96578777088,"internally_reserved":0,"allocated":21925888,"data_stored":26077740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.318},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.505}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.373},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.88600000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.626}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35128,"kb_used_data":4012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94336712,"statfs":{"total":96636764160,"available":96600793088,"internally_reserved":0,"allocated":4108288,"data_stored":4087910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.2649999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.439},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93600000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.40699999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.501},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.84399999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44468,"kb_used_data":9256,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94327372,"statfs":{"total":96636764160,"available":96591228928,"internally_reserved":0,"allocated":9478144,"data_stored":10581923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.497}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.78700000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88400000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.006}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.012},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.1080000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.208},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48628,"kb_used_data":13412,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94323212,"statfs":{"total":96636764160,"available":96586969088,"internally_reserved":0,"allocated":13733888,"data_stored":17230256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":3.4870000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.038}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.86499999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.70299999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.099}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.173},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12382208,"data_stored":16299309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8704000,"data_stored":10235520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3338240,"data_stored":3741526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21155840,"data_stored":25731356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15245312,"data_stored":15349324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11632640,"data_stored":14011335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5980160,"data_stored":6020934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16965632,"data_stored":19166524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:33:46.745 INFO:tasks.thrashosds.thrasher:Setting 3.10 to [7, 5] 2024-09-21T15:33:46.746 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.10', '7', '5'] 2024-09-21T15:33:46.746 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.10 7 5 2024-09-21T15:33:46.768 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:46.769+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:46.817 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:46.818 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:46.832 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:46.876 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:46.877+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:46.969 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:46.967+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:47.049 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:47.049 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:47.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:47.116 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:47.117+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:47.170 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:47.169+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:47.199 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:47.199 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:47.217 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:47.271 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:47.267+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:47.372 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:47.369+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:47.407 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:47.407 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:47.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:47.473 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:47.471+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:47.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:47.573 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:47.573+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:47.591 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:47.591 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:47.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:47.663 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:47.663 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:47.674 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:47.671+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:47.679 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:47.679 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:47.696 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:47.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:47.730 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:47.747 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:47.748 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:47.764 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:47.765 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:47.775 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:47.771+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:47.781 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:47.781 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:47.796 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:47.796 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.008729360999993, 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.797 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:47.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:47.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:47.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 69.002118155000005, 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:47.803 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:47.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:47.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.427290436, 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.809 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:47.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:47.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:47.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 54.125476442999997, 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:47.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:47.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 54.046659789000003, 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:47.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 54.016306776, 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.824 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.829 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 54.008435081000002, 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:47.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.997255633000002, 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:47.835 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:47.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:47.840 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.992023267, 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:47.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.973537985999997, 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:47.845 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:47.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.846384483999998, 2024-09-21T15:33:47.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:47.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:47.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.846228162000003, 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.855 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:47.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.840545937000002, 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:47.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.866 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 53.839583118, 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.870 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.078738195, 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.871 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:47.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 39.769713561000003, 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:47.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:47.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:47.882 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.490648583999999, 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:47.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:47.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 31.159809086999999, 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.889 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:47.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:47.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:47.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26861 3.1c 3:3bbf239b:::rbd_data.1191e6bca1cd.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.4899606529999998, 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0087661500000000003, 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26861 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306150+0000", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307085+0000", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000935209 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307123+0000", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8003999999999997e-05 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307141+0000", 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7842e-05 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307265+0000", 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124389 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307530+0000", 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026449399999999999 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307916+0000", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038618499999999999 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314315+0000", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063993590000000003 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314679+0000", 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036339 2024-09-21T15:33:47.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314820+0000", 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014107400000000001 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314919+0000", 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9418000000000003e-05 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26864 3.1c 3:3c39757f:::rbd_data.1191e6bca1cd.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 7.4866667400000004, 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077293780000000003, 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26864 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309444+0000", 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:47.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310084+0000", 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00063922000000000002 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310138+0000", 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4676000000000003e-05 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:47.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310160+0000", 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1325999999999999e-05 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310956+0000", 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00079680599999999995 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311180+0000", 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337600000000001 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311734+0000", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055437499999999996 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.315365+0000", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036305460000000001 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.316842+0000", 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001476904 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317088+0000", 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024673799999999998 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317176+0000", 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8107000000000002e-05 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:47.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:47.907 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:47.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:47.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:47.909 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:47.905+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:set 3.10 pg_upmap mapping to [7,5] 2024-09-21T15:33:48.009 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:48.007+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:48.095 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:48.096 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:48.096 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:48.096 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:48.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:48.111 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:48.109+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:48.211 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:48.209+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:48.296 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:48.296 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:48.296 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:48.296 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:48.296 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:48.312 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:48.307+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:48.313 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:48.412 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:48.409+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:48.501 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 50.038209055000003, 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:48.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:48.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:48.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.807931678999999, 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:48.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:48.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:48.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.506560888000003, 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:48.512 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:48.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 36.831047067999997, 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.518 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:48.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:48.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:48.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:48.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.523 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.502190706, 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:48.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.502174214, 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:48.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.502157842000001, 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:48.532 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.536 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.502141177, 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.502124354999999, 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:48.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:48.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501796436999999, 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:48.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.549 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501765795000001, 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:48.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.553 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501747651000001, 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:48.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:48.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501717194999999, 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.558 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:48.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.562 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501698907, 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:48.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:48.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501681618999999, 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.567 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:48.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:48.571 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501664953000001, 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501640606, 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501624464000001, 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:48.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501607919, 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:48.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 14.501580375, 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:48.589 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:48.593 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:48.594 INFO:teuthology.orchestra.run.smithi107.stdout:2811 read 0x33537ac thru 0x335aeee (0x7743 bytes) 2024-09-21T15:33:48.595 INFO:teuthology.orchestra.run.smithi107.stdout:2812 trunc from 0x92a286d to 0x256af9d 2024-09-21T15:33:48.595 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:48.596 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:48.593+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:48.613 INFO:teuthology.orchestra.run.smithi107.stdout:2814 punch from 0x17315ec to 0x1739fb8, (0x89cc bytes) 2024-09-21T15:33:48.631 INFO:teuthology.orchestra.run.smithi107.stdout:2817 writesame 0x613afd3 thru 0x614a4a7 data_size 0x1(0xf4d5 bytes) 2024-09-21T15:33:48.640 INFO:teuthology.orchestra.run.smithi107.stdout:2819 trunc from 0x614a4a8 to 0x2baf9a4 2024-09-21T15:33:48.688 INFO:teuthology.orchestra.run.smithi107.stdout:2820 write 0xdcfbd19 thru 0xdd02359 (0x6641 bytes) 2024-09-21T15:33:48.695 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:48.691+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:48.696 INFO:teuthology.orchestra.run.smithi107.stdout:2821 write 0x88f227c thru 0x88f38f6 (0x167b bytes) 2024-09-21T15:33:48.703 INFO:teuthology.orchestra.run.smithi107.stdout:2822 writesame 0x724f187 thru 0x72501ca data_size 0x6(0x1044 bytes) 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:2823 read 0x7280382 thru 0x7289954 (0x95d3 bytes) 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:2826 read 0x75ae3ca thru 0x75b8c65 (0xa89c bytes) 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:2828 read 0x8d945ed thru 0x8d9511c (0xb30 bytes) 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:2829 read 0x662ff59 thru 0x66390e1 (0x9189 bytes) 2024-09-21T15:33:48.713 INFO:teuthology.orchestra.run.smithi107.stdout:2830 write 0x2468c6f thru 0x246fba9 (0x6f3b bytes) 2024-09-21T15:33:48.719 INFO:teuthology.orchestra.run.smithi107.stdout:2831 punch from 0x9b49686 to 0x9b549c2, (0xb33c bytes) 2024-09-21T15:33:48.719 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:48.719 INFO:teuthology.orchestra.run.smithi107.stdout:2833 read 0x589efd9 thru 0x58abd73 (0xcd9b bytes) 2024-09-21T15:33:48.720 INFO:teuthology.orchestra.run.smithi107.stdout:2834 write 0x43fc1ed thru 0x43fe73a (0x254e bytes) 2024-09-21T15:33:48.726 INFO:teuthology.orchestra.run.smithi107.stdout:2835 write 0x72d57d6 thru 0x72d8165 (0x2990 bytes) 2024-09-21T15:33:48.733 INFO:teuthology.orchestra.run.smithi107.stdout:2836 read 0x3dce6af thru 0x3dcf63a (0xf8c bytes) 2024-09-21T15:33:48.733 INFO:teuthology.orchestra.run.smithi107.stdout:2838 write 0xbd560f0 thru 0xbd5fbfd (0x9b0e bytes) 2024-09-21T15:33:48.741 INFO:teuthology.orchestra.run.smithi107.stdout:2839 punch from 0xb960a8 to 0xb9ee58, (0x8db0 bytes) 2024-09-21T15:33:48.741 INFO:teuthology.orchestra.run.smithi107.stdout:2840 read 0x6d9be5 thru 0x6e9b7a (0xff96 bytes) 2024-09-21T15:33:48.749 INFO:teuthology.orchestra.run.smithi107.stdout:2841 trunc from 0xdd0235a to 0xb893d5e 2024-09-21T15:33:48.766 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:48.766 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:48.771 INFO:teuthology.orchestra.run.smithi107.stdout:2842 read 0x5b830ed thru 0x5b89d53 (0x6c67 bytes) 2024-09-21T15:33:48.771 INFO:teuthology.orchestra.run.smithi107.stdout:2843 read 0x450ae64 thru 0x450d31b (0x24b8 bytes) 2024-09-21T15:33:48.776 INFO:teuthology.orchestra.run.smithi107.stdout:2848 trunc from 0xb893d5e to 0xdbae9eb 2024-09-21T15:33:48.780 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:48.796 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:48.795+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:48.799 INFO:teuthology.orchestra.run.smithi107.stdout:2849 write 0xdbe7402 thru 0xdbf0cc5 (0x98c4 bytes) 2024-09-21T15:33:48.811 INFO:teuthology.orchestra.run.smithi107.stdout:2850 write 0x4d8537 thru 0x4dfd1d (0x77e7 bytes) 2024-09-21T15:33:48.816 INFO:teuthology.orchestra.run.smithi107.stdout:2851 writesame 0xc3d6763 thru 0xc3dd914 data_size 0x36(0x71b2 bytes) 2024-09-21T15:33:48.825 INFO:teuthology.orchestra.run.smithi107.stdout:2852 read 0x39b18bf thru 0x39bc36c (0xaaae bytes) 2024-09-21T15:33:48.825 INFO:teuthology.orchestra.run.smithi107.stdout:2853 write 0x32cc150 thru 0x32dac63 (0xeb14 bytes) 2024-09-21T15:33:48.837 INFO:teuthology.orchestra.run.smithi107.stdout:2855 punch from 0xd40a5f to 0xd45798, (0x4d39 bytes) 2024-09-21T15:33:48.837 INFO:teuthology.orchestra.run.smithi107.stdout:2856 read 0x7eedaf4 thru 0x7efbbb0 (0xe0bd bytes) 2024-09-21T15:33:48.837 INFO:teuthology.orchestra.run.smithi107.stdout:2857 write 0x66242fa thru 0x662fd4b (0xba52 bytes) 2024-09-21T15:33:48.844 INFO:teuthology.orchestra.run.smithi107.stdout:2858 read 0xbbe551f thru 0xbbec973 (0x7455 bytes) 2024-09-21T15:33:48.844 INFO:teuthology.orchestra.run.smithi107.stdout:2860 write 0x5ad94b7 thru 0x5ad9c78 (0x7c2 bytes) 2024-09-21T15:33:48.851 INFO:teuthology.orchestra.run.smithi107.stdout:2861 read 0x470b4b4 thru 0x470c36a (0xeb7 bytes) 2024-09-21T15:33:48.851 INFO:teuthology.orchestra.run.smithi107.stdout:2863 read 0x978e90b thru 0x979038f (0x1a85 bytes) 2024-09-21T15:33:48.863 INFO:teuthology.orchestra.run.smithi107.stdout:2864 trunc from 0xdbf0cc6 to 0x94b69ec 2024-09-21T15:33:48.883 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:33:48.894 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:48.894 INFO:teuthology.orchestra.run.smithi107.stdout:2866 write 0x15e365e thru 0x15e6766 (0x3109 bytes) 2024-09-21T15:33:48.896 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:48.897+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:48.898 INFO:teuthology.orchestra.run.smithi107.stdout:2867 write 0x6ce94db thru 0x6cf3f5b (0xaa81 bytes) 2024-09-21T15:33:48.905 INFO:teuthology.orchestra.run.smithi107.stdout:2868 write 0x62bf68e thru 0x62c6625 (0x6f98 bytes) 2024-09-21T15:33:48.909 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:48.909 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:48.909 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:48.909 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:48.909 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:48.910 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:48.920 INFO:teuthology.orchestra.run.smithi107.stdout:2870 write 0xaf39036 thru 0xaf423fa (0x93c5 bytes) 2024-09-21T15:33:48.924 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:48.926 INFO:teuthology.orchestra.run.smithi107.stdout:2872 write 0x1948fc8 thru 0x19513c6 (0x83ff bytes) 2024-09-21T15:33:48.934 INFO:teuthology.orchestra.run.smithi107.stdout:2873 trunc from 0xaf423fb to 0xa3f21f1 2024-09-21T15:33:48.950 INFO:teuthology.orchestra.run.smithi107.stdout:2874 punch from 0x293944a to 0x293b0b4, (0x1c6a bytes) 2024-09-21T15:33:48.950 INFO:teuthology.orchestra.run.smithi107.stdout:2876 clone 23 order 21 su 16384 sc 11 2024-09-21T15:33:49.016 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:49.017+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47133 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9123840~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0288728640000002, 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091148090000000001, 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.093 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47133 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063911+0000", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063916+0000", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6110000000000004e-06 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063928+0000", 2024-09-21T15:33:49.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1948e-05 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063937+0000", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2930000000000006e-06 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.154776+0000", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090839016999999994 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.154905+0000", 2024-09-21T15:33:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012873299999999999 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155060+0000", 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000155205 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47134 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9127936~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0288392369999997, 2024-09-21T15:33:49.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091431745999999994, 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47134 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063945+0000", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063949+0000", 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.391e-06 2024-09-21T15:33:49.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063961+0000", 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1888e-05 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063972+0000", 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1027e-05 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155106+0000", 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091133567999999998 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155229+0000", 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012347899999999999 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155378+0000", 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014811300000000001 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47135 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9132032~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0288037210000001, 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091711168999999995, 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47135 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063985+0000", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6940000000000003e-06 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063997+0000", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2152e-05 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064007+0000", 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1670000000000005e-06 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155422+0000", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091415095000000002 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155548+0000", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012659300000000001 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155692+0000", 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014420500000000001 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.105 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47136 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9136128~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0287696559999997, 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091991677999999993, 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47136 2024-09-21T15:33:49.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064019+0000", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3579999999999996e-06 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064031+0000", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1941e-05 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064040+0000", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599999999999997e-06 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155738+0000", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091698084999999999 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155862+0000", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012419300000000001 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156007+0000", 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014476300000000001 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47137 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9140224~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0286584469999998, 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092194095000000004, 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.110 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47137 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:49.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064048+0000", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999208 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064133+0000", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5834000000000005e-05 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064152+0000", 2024-09-21T15:33:49.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8505e-05 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064165+0000", 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2853e-05 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156048+0000", 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091882894000000007 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156168+0000", 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120517 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156321+0000", 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015246500000000001 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47140 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9414656~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:49.114 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0263887990000002, 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091029811000000002, 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47140 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:49.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066395+0000", 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066400+0000", 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8180000000000003e-06 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066412+0000", 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1888999999999999e-05 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066422+0000", 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0355e-05 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157158+0000", 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090735646000000003 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157285+0000", 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012709599999999999 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157426+0000", 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000140906 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47141 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9418752~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0263549520000002, 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091333279000000003, 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47141 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:49.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066429+0000", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066434+0000", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0000000000000004e-06 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066447+0000", 2024-09-21T15:33:49.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2128e-05 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066457+0000", 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9820000000000003e-06 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157486+0000", 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091029652000000003 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157617+0000", 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013117000000000001 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157763+0000", 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000146012 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47142 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9422848~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.123 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.026319365, 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091614438000000006, 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47142 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:49.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066465+0000", 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066470+0000", 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6909999999999996e-06 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066481+0000", 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.182e-05 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066492+0000", 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0169999999999999e-05 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157814+0000", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091322540999999993 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157937+0000", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012265 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158080+0000", 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014326200000000001 2024-09-21T15:33:49.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47143 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9426944~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.026277179, 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092001538999999993, 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47143 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066507+0000", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066512+0000", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4160000000000003e-06 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066525+0000", 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2383e-05 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066535+0000", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0247e-05 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158180+0000", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091645236000000005 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158334+0000", 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015351199999999999 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158509+0000", 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017578299999999999 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47152 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8547328~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0223866069999996, 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091150810999999998, 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47152 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070401+0000", 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.542e-06 2024-09-21T15:33:49.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070407+0000", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0870000000000003e-06 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070412+0000", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1189999999999996e-06 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161282+0000", 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090869968999999995 2024-09-21T15:33:49.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161405+0000", 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012354999999999999 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161549+0000", 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000143969 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.136 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47153 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8551424~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0223686369999996, 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091466130000000007, 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47153 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070419+0000", 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6139999999999999e-06 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070428+0000", 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4700000000000008e-06 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070433+0000", 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2399999999999998e-06 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161601+0000", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091167699000000005 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161735+0000", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013359299999999999 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161883+0000", 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014793900000000001 2024-09-21T15:33:49.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47163 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9116672~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.018354521, 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090996833999999999, 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.141 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47163 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074430+0000", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074433+0000", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8329999999999998e-06 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074439+0000", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0870000000000003e-06 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074444+0000", 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9690000000000002e-06 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165147+0000", 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090702609000000003 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165283+0000", 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013657 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165427+0000", 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014419199999999999 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47164 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9120768~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0183365100000001, 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091301265000000006, 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.145 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47164 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074448+0000", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074451+0000", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5959999999999998e-06 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074458+0000", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5390000000000002e-06 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074463+0000", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1420000000000001e-06 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165477+0000", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091014034999999993 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165604+0000", 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012615700000000001 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165750+0000", 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014622499999999999 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47175 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9427968~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.149 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0142585899999998, 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091182622000000005, 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47175 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:49.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078529+0000", 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.802e-06 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078536+0000", 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2370000000000003e-06 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078541+0000", 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4090000000000002e-06 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169371+0000", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090829441999999996 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169528+0000", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015705800000000001 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169709+0000", 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018110399999999999 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.153 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47204 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9461760~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0099072930000004, 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090974816, 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47204 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082877+0000", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082881+0000", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1219999999999997e-06 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082895+0000", 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3621999999999999e-05 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082903+0000", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5229999999999995e-06 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173578+0000", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090674169999999998 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173710+0000", 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013205000000000001 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173853+0000", 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014296900000000001 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47207 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9465856~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0097974839999999, 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091198736000000002, 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47207 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082910+0000", 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999228 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082994+0000", 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3819000000000005e-05 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.083009+0000", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4645e-05 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.083020+0000", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1024e-05 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173904+0000", 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090884548999999995 2024-09-21T15:33:49.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.174027+0000", 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122583 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.174186+0000", 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015947199999999999 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47229 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10035200~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0059823049999999, 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091206161999999993, 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47229 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086802+0000", 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086807+0000", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7659999999999998e-06 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086818+0000", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0997e-05 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086828+0000", 2024-09-21T15:33:49.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0729e-05 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.177737+0000", 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090908364000000005 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.177868+0000", 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000131283 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.178009+0000", 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014094300000000001 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47240 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10342400~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.001933223, 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090925040999999998, 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.167 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47240 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090851+0000", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090855+0000", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2220000000000004e-06 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090868+0000", 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3301e-05 2024-09-21T15:33:49.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090878+0000", 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181499+0000", 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090620777999999999 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181632+0000", 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000133332 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181777+0000", 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014491999999999999 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47241 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10346496~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:49.171 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4.0018992530000004, 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091328454000000003, 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47241 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090885+0000", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090889+0000", 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2320000000000003e-06 2024-09-21T15:33:49.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090900+0000", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0796e-05 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090909+0000", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4530000000000002e-06 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181862+0000", 2024-09-21T15:33:49.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090953645999999999 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.182018+0000", 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015544500000000001 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.182214+0000", 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019647600000000001 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47251 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9733120~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.998079374, 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090993061, 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47251 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.176 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094705+0000", 2024-09-21T15:33:49.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094708+0000", 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8890000000000002e-06 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094717+0000", 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5310000000000001e-06 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094722+0000", 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3929999999999999e-06 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185367+0000", 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090644481999999998 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185521+0000", 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154661 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:49.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185699+0000", 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000177534 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:49.180 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:49.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:49.182 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:49.179+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:49.282 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:49.281+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:49.312 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:49.312 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:49.312 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:49.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:49.383 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:49.379+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:49.443 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:49.443 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:49.443 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:49.443 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:49.443 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:49.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:49.500 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:49.499+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:49.585 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:49.585+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 50.908411211999997, 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:49.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:49.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.626 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 50.898088917000003, 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:49.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:49.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 47.970591691999999, 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:49.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 47.936680258000003, 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:49.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10948 3.1 3:82fda950:::rbd_data.11917b2c111e.0000000000000011:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.936717528000003, 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025101894999999999, 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.640 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10948 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681359+0000", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681370+0000", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.135e-05 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681391+0000", 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0633e-05 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681400+0000", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8109999999999992e-06 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681492+0000", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2101000000000005e-05 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681709+0000", 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021704800000000001 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.687063+0000", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053539759999999999 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706359+0000", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019296002 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706374+0000", 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5245999999999999e-05 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706424+0000", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9996e-05 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706463+0000", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9069999999999997e-05 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11973 3.18 3:1bdac4bd:::rbd_data.11917b2c111e.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:49.645 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.516750256000002, 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019588250000000001, 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11973 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101326+0000", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101338+0000", 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2381000000000001e-05 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101361+0000", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2884e-05 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101372+0000", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0716999999999999e-05 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101427+0000", 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4626000000000002e-05 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101493+0000", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6115e-05 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.104640+0000", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003147501 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120784+0000", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016143372999999999 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:49.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120799+0000", 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5832999999999999e-05 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120871+0000", 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1977000000000006e-05 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120917+0000", 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5269999999999999e-05 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16568 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629500021, 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016067491999999999, 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16568 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988579+0000", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988578+0000", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988580+0000", 2024-09-21T15:33:49.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.432e-06 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988584+0000", 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.765e-06 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988586+0000", 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.599e-06 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004385+0000", 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015799359999999998 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004428+0000", 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2552000000000002e-05 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004646+0000", 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021814800000000001 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.654 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16569 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.62947973, 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016416218, 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16569 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988600+0000", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3519999999999999e-06 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988603+0000", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4070000000000002e-06 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988606+0000", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7350000000000001e-06 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004678+0000", 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016071621000000001 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004772+0000", 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4575999999999997e-05 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005015+0000", 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000242752 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16570 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629470298999999, 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016891402999999999, 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16570 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988609+0000", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3659999999999999e-06 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988612+0000", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988615+0000", 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5160000000000001e-06 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005071+0000", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016456487999999998 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005205+0000", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133638 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005500+0000", 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029470199999999998 2024-09-21T15:33:49.662 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16571 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629461730999999, 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017340120000000001, 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.663 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16571 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988618+0000", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2559999999999999e-06 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988622+0000", 2024-09-21T15:33:49.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.087e-06 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988626+0000", 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1559999999999997e-06 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005524+0000", 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016897760000000001 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005594+0000", 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9998000000000003e-05 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005957+0000", 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036308 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16572 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.667 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629448837, 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017681184999999999, 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16572 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988632+0000", 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.227e-06 2024-09-21T15:33:49.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988638+0000", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1990000000000003e-06 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988641+0000", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.835e-06 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005986+0000", 2024-09-21T15:33:49.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017345098999999999 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006062+0000", 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5685999999999993e-05 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006311+0000", 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000249452 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16574 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629435609, 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956159999999999, 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16574 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988644+0000", 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.35e-06 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988647+0000", 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1750000000000001e-06 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988650+0000", 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.553e-06 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006336+0000", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017685671 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006376+0000", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0154999999999998e-05 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006599+0000", 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022348800000000001 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16575 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629425327, 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01820097, 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16575 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988654+0000", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1930000000000001e-06 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988657+0000", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.016e-06 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988661+0000", 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2200000000000001e-06 2024-09-21T15:33:49.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006625+0000", 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017964285 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006669+0000", 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4363000000000002e-05 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006854+0000", 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000185105 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16723 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629051193, 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016156074999999999, 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16723 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989027+0000", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989029+0000", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6929999999999999e-06 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989033+0000", 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3479999999999997e-06 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989036+0000", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.565e-06 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005031+0000", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015995019999999999 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005095+0000", 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4395999999999998e-05 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005184+0000", 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8259999999999999e-05 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16726 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629040846000001, 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016361140999999999, 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16726 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989039+0000", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.114e-06 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989062+0000", 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2787000000000001e-05 2024-09-21T15:33:49.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989066+0000", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005228+0000", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016161933999999999 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005298+0000", 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0092999999999995e-05 2024-09-21T15:33:49.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005399+0000", 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010131700000000001 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16729 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.629010137, 2024-09-21T15:33:49.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016513967000000001, 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16729 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:49.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989068+0000", 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989070+0000", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.809e-06 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989076+0000", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0530000000000003e-06 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989083+0000", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9450000000000004e-06 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005427+0000", 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016344145000000001 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005495+0000", 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8040999999999994e-05 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005583+0000", 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7250000000000007e-05 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16732 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.628992411, 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016636597999999999, 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16732 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989091+0000", 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4739999999999996e-06 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989097+0000", 2024-09-21T15:33:49.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9909999999999999e-06 2024-09-21T15:33:49.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989102+0000", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.792e-06 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005607+0000", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016505407 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005670+0000", 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2288999999999999e-05 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005723+0000", 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3154e-05 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16735 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.628972723, 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016763145, 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16735 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989105+0000", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989108+0000", 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7149999999999998e-06 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989123+0000", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5291999999999999e-05 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989128+0000", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005749+0000", 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016620945000000002 2024-09-21T15:33:49.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005806+0000", 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6436e-05 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005869+0000", 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3303e-05 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16738 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.628945817, 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016893127000000001, 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16738 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989132+0000", 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989135+0000", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4530000000000001e-06 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989141+0000", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0599999999999996e-06 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989146+0000", 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:33:49.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005907+0000", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016761426999999999 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005977+0000", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0178000000000005e-05 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006026+0000", 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8862000000000001e-05 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.707 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16740 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.628929165000001, 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016975200999999999, 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16740 2024-09-21T15:33:49.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989149+0000", 2024-09-21T15:33:49.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989151+0000", 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2170000000000001e-06 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989157+0000", 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.631e-06 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989161+0000", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4700000000000004e-06 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006040+0000", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016878955000000001 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006077+0000", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6838999999999997e-05 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:49.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006125+0000", 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7547999999999999e-05 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:49.712 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:49.713 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:49.714 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:49.711+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:49.814 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:49.813+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:49.884 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:49.884 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:49.884 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:49.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:49.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:49.915 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:49.913+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:50.017 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:50.017+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:50.039 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:50.039 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:50.039 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:50.039 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:50.039 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:50.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:50.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:50.101 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:50.118 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:50.117+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:50.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:50.137 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:50.137 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:50.145 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T15:33:50.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:50.155 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:50.175 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:50.176 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:50.196 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:50.196 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:50.220 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:50.222 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:50.221+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:50.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:50.262 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:50.332 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:50.331+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:50.405 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:50.405 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:50.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:50.421 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:50.421+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:50.521 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:50.519+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:50.600 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:50.600 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:50.614 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:50.656 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:50.655+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:50.723 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:50.719+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:50.787 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:50.787 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:50.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:50.824 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:50.825+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:50.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:33:50.926 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:50.925+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:50.975 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:50.975 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:50.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:51.026 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:51.025+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:51.127 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:51.123+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:51.138 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:51.138 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:51.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:51.240 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:51.239+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:51.329 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:51.327+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:51.342 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:51.342 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:51.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:51.431 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:51.427+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:51.514 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:51.514 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:51.529 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:51.531 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:51.529+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:51.633 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:51.633+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:51.656 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:51.656 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:51.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:51.733 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:51.731+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:51.834 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:51.833+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:51.839 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:51.840 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:51.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:51.907 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:33:51.952 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:51.953+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:52.036 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:52.033+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:52.046 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:52.046 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:52.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:52.157 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:52.157+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:52.221 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:52.222 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:52.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:52.238 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:52.235+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:52.289 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:52.339 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:52.335+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:52.390 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:52.390 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:52.404 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:52.440 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:52.435+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:52.470 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:52.470 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:52.487 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:52.487 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:52.504 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:52.504 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:52.513 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:52.513 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:52.520 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:52.521 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:52.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:52.538 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:52.538 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:52.540 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:52.539+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:52.556 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:52.556 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:52.574 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:52.574 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:52.590 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:52.591 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:52.641 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:52.641+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:52.657 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:52.657 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:52.657 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:52.657 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:52.658 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:52.675 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:52.743 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:52.739+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 54.375399422999998, 2024-09-21T15:33:52.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:52.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.843 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.145122047000001, 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.844 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:52.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:52.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.843751255999997, 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.850 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:52.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:52.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.168237435999998, 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:52.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:52.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:52.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.861 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.839381073999999, 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:52.862 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.863 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.864 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.865 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.839364582000002, 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.866 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:52.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.839348210000001, 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:52.870 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.874 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.839331545, 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:52.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.839314723000001, 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.879 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:52.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838986805000001, 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:52.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838956162999999, 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.888 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:52.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.892 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838938019, 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838907562999999, 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.897 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:52.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:52.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838889275, 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838871987000001, 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.906 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:52.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:52.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838855321, 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:52.910 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.916 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838830974, 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:52.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838814832000001, 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.921 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:52.925 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838798286999999, 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 18.838770743000001, 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:52.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:52.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.933 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:52.934 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:52.934 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:52.934 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:52.934 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:52.934 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T15:33:52.935 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:33:52.936 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:52.936 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:52.933+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:53.036 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:53.033+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:53.133 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:85997 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.098627+0000", 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.023327437999999999, 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023353635000000001, 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.134 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 85997 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098627+0000", 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098627+0000", 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098627+0000", 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098629+0000", 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3379999999999999e-06 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098635+0000", 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.098640+0000", 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5879999999999999e-06 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86060 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099257+0000", 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022696992999999999, 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022763813000000001, 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.137 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86060 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099257+0000", 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099257+0000", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099257+0000", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099259+0000", 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2910000000000002e-06 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099266+0000", 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2149999999999998e-06 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099270+0000", 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5560000000000001e-06 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.140 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86100 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099869+0000", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022085007, 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022182515, 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86100 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099869+0000", 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099869+0000", 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099869+0000", 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099871+0000", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3970000000000001e-06 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099877+0000", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.908e-06 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099882+0000", 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6210000000000003e-06 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.143 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86132 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100522+0000", 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021432769000000001, 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021560525000000001, 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86132 2024-09-21T15:33:53.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100522+0000", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100522+0000", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100521+0000", 2024-09-21T15:33:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100524+0000", 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4360000000000001e-06 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100529+0000", 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8420000000000001e-06 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100535+0000", 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5210000000000002e-06 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86164 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101418+0000", 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020536796, 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.020732272, 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.147 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86164 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101418+0000", 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101418+0000", 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101417+0000", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101420+0000", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8159999999999998e-06 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101426+0000", 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4600000000000002e-06 2024-09-21T15:33:53.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101430+0000", 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5940000000000004e-06 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86324 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114633+0000", 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0073216089999999998, 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.007579466, 2024-09-21T15:33:53.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86324 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114633+0000", 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114633+0000", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114633+0000", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114635+0000", 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1019999999999999e-06 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114641+0000", 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0900000000000001e-06 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114645+0000", 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6630000000000001e-06 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86356 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.153 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115315+0000", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0066391289999999997, 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069544560000000004, 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86356 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115315+0000", 2024-09-21T15:33:53.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115315+0000", 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115315+0000", 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115317+0000", 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.182e-06 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115323+0000", 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4709999999999998e-06 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115327+0000", 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5909999999999998e-06 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.156 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86388 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118814+0000", 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0031405949999999999, 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0035092809999999999, 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86388 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118814+0000", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118814+0000", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118813+0000", 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118816+0000", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2450000000000001e-06 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118821+0000", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3929999999999999e-06 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118826+0000", 2024-09-21T15:33:53.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4380000000000003e-06 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86420 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119575+0000", 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0023790920000000002, 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0027772420000000001, 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.160 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86420 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119575+0000", 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119575+0000", 2024-09-21T15:33:53.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119575+0000", 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119577+0000", 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1119999999999999e-06 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119582+0000", 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.434e-06 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119587+0000", 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2599999999999999e-06 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86062 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.163 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099276+0000", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022678753999999999, 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023109905, 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86062 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099276+0000", 2024-09-21T15:33:53.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099276+0000", 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099275+0000", 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099278+0000", 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5160000000000001e-06 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099284+0000", 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0039999999999996e-06 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099291+0000", 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.166 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86325 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114650+0000", 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0073048870000000004, 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077651990000000004, 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86325 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.167 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114650+0000", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114650+0000", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114649+0000", 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114652+0000", 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7049999999999999e-06 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114657+0000", 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6520000000000003e-06 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114662+0000", 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7210000000000002e-06 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86357 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118178+0000", 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.003776836, 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042659960000000002, 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86357 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118178+0000", 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118178+0000", 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115331+0000", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9971533 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118181+0000", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0028507519999999998 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118191+0000", 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3170000000000008e-06 2024-09-21T15:33:53.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118198+0000", 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0199999999999997e-06 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86389 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118830+0000", 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.003124211, 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036433530000000002, 2024-09-21T15:33:53.173 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86389 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118830+0000", 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118830+0000", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118830+0000", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118832+0000", 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1469999999999999e-06 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118837+0000", 2024-09-21T15:33:53.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4229999999999996e-06 2024-09-21T15:33:53.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118842+0000", 2024-09-21T15:33:53.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5469999999999998e-06 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86421 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119590+0000", 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0023640219999999999, 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0029335199999999998, 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86421 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119590+0000", 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119590+0000", 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119590+0000", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119592+0000", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.041e-06 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119598+0000", 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4469999999999997e-06 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119602+0000", 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5210000000000004e-06 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86063 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099296+0000", 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022658711000000002, 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023285472000000002, 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.181 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86063 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099296+0000", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099296+0000", 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099295+0000", 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099307+0000", 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1755e-05 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099314+0000", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5830000000000003e-06 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099319+0000", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1329999999999998e-06 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86102 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099902+0000", 2024-09-21T15:33:53.184 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022052572999999999, 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022752503, 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86102 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099902+0000", 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099902+0000", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099902+0000", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099904+0000", 2024-09-21T15:33:53.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0930000000000001e-06 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099909+0000", 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5280000000000003e-06 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099914+0000", 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5519999999999998e-06 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86134 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100557+0000", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021397731999999999, 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022158964, 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86134 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100557+0000", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100557+0000", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100556+0000", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100559+0000", 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3350000000000001e-06 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100564+0000", 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5149999999999997e-06 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100568+0000", 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3939999999999998e-06 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.190 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86166 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101462+0000", 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020492468, 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021283347000000001, 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86166 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.191 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101462+0000", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101462+0000", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101462+0000", 2024-09-21T15:33:53.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101464+0000", 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6199999999999999e-06 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101470+0000", 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3319999999999997e-06 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101475+0000", 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6460000000000001e-06 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86326 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114666+0000", 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0072885299999999997, 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0081112360000000008, 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86326 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114666+0000", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114666+0000", 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114666+0000", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114677+0000", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1786e-05 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114683+0000", 2024-09-21T15:33:53.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.981e-06 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114688+0000", 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6670000000000004e-06 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86358 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118217+0000", 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0037379000000000002, 2024-09-21T15:33:53.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00460276, 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86358 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118217+0000", 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118217+0000", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118216+0000", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118219+0000", 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5429999999999998e-06 2024-09-21T15:33:53.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118226+0000", 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5100000000000004e-06 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118231+0000", 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.921e-06 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86390 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118846+0000", 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0031086009999999999, 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0040316400000000004, 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86390 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118846+0000", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118846+0000", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118846+0000", 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118848+0000", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0159999999999998e-06 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118853+0000", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5230000000000003e-06 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118857+0000", 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3789999999999999e-06 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.203 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86422 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122163+0000", 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9997921, 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00077407300000000001, 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86422 2024-09-21T15:33:53.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122163+0000", 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122163+0000", 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119605+0000", 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9974427 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122166+0000", 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025610049999999999 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122176+0000", 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8430000000000005e-06 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122184+0000", 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6380000000000004e-06 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86065 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099324+0000", 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022630902000000001, 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023671933999999999, 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.207 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86065 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099324+0000", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099324+0000", 2024-09-21T15:33:53.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099323+0000", 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099326+0000", 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4820000000000001e-06 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099332+0000", 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3910000000000003e-06 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099338+0000", 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6679999999999998e-06 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86327 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114692+0000", 2024-09-21T15:33:53.210 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0072622950000000002, 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083445120000000001, 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86327 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114692+0000", 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114692+0000", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114692+0000", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114694+0000", 2024-09-21T15:33:53.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.204e-06 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114699+0000", 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1499999999999998e-06 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114704+0000", 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6709999999999998e-06 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86359 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118235+0000", 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0037190819999999999, 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0048317799999999999, 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86359 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118235+0000", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118235+0000", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118235+0000", 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118237+0000", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.306e-06 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118243+0000", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8329999999999999e-06 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118248+0000", 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7299999999999996e-06 2024-09-21T15:33:53.216 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86391 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118862+0000", 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0030929180000000001, 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0042335589999999996, 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.217 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86391 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118862+0000", 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118862+0000", 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118861+0000", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118863+0000", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4119999999999999e-06 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118870+0000", 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0870000000000003e-06 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118874+0000", 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.634e-06 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86423 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122190+0000", 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9997644, 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00093356299999999999, 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.220 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86423 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122190+0000", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122190+0000", 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122190+0000", 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122197+0000", 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2470000000000002e-06 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122203+0000", 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1020000000000002e-06 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122208+0000", 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3000000000000001e-06 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86067 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.223 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099342+0000", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022612127999999999, 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023811051999999999, 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86067 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099342+0000", 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099342+0000", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099342+0000", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099344+0000", 2024-09-21T15:33:53.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2800000000000002e-06 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099350+0000", 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0220000000000001e-06 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099355+0000", 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7639999999999996e-06 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86104 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099935+0000", 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.02201966, 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023255794999999999, 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86104 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099935+0000", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099935+0000", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099934+0000", 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099937+0000", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2460000000000002e-06 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099943+0000", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8610000000000003e-06 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099947+0000", 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3800000000000004e-06 2024-09-21T15:33:53.229 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86136 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100600+0000", 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021354583, 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022647238, 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86136 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100600+0000", 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100600+0000", 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100599+0000", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100602+0000", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3439999999999999e-06 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100608+0000", 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.942e-06 2024-09-21T15:33:53.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100612+0000", 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3510000000000004e-06 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86168 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.102201+0000", 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.019753729000000001, 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021104392999999999, 2024-09-21T15:33:53.233 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86168 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102201+0000", 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102201+0000", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102200+0000", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102203+0000", 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7269999999999999e-06 2024-09-21T15:33:53.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102210+0000", 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0199999999999997e-06 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102215+0000", 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7820000000000001e-06 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86328 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114708+0000", 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0072460809999999997, 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0086542549999999996, 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86328 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114708+0000", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114708+0000", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114708+0000", 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114711+0000", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7140000000000002e-06 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114716+0000", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5659999999999997e-06 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114721+0000", 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6059999999999996e-06 2024-09-21T15:33:53.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86360 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118252+0000", 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0037027599999999998, 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0051528880000000004, 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.240 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86360 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118252+0000", 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118252+0000", 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118251+0000", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118253+0000", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.137e-06 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118259+0000", 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9889999999999997e-06 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118264+0000", 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6530000000000002e-06 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86392 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118893+0000", 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.003061065, 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045402519999999998, 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.243 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86392 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118893+0000", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118893+0000", 2024-09-21T15:33:53.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118893+0000", 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118896+0000", 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7039999999999999e-06 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118902+0000", 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8299999999999998e-06 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118907+0000", 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8069999999999999e-06 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86424 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122213+0000", 2024-09-21T15:33:53.246 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9997416, 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001249479, 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86424 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122213+0000", 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122213+0000", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122212+0000", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122215+0000", 2024-09-21T15:33:53.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.384e-06 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122220+0000", 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8200000000000002e-06 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122226+0000", 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1619999999999999e-06 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86069 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099359+0000", 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022595150000000001, 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024140036, 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86069 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099359+0000", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099359+0000", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099359+0000", 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099361+0000", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2709999999999999e-06 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099367+0000", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0050000000000001e-06 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099372+0000", 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.369e-06 2024-09-21T15:33:53.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86329 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114725+0000", 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0072298930000000003, 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0088083780000000004, 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.253 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86329 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114725+0000", 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114725+0000", 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114724+0000", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114726+0000", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0829999999999998e-06 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114732+0000", 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4369999999999998e-06 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114736+0000", 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4599999999999996e-06 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86361 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118268+0000", 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0036861559999999999, 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.005325392, 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.256 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86361 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118268+0000", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118268+0000", 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118268+0000", 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118270+0000", 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118276+0000", 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5799999999999999e-06 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118282+0000", 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4559999999999997e-06 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86393 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.259 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118911+0000", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0030431500000000001, 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0047405829999999996, 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86393 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118911+0000", 2024-09-21T15:33:53.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118911+0000", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118911+0000", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118913+0000", 2024-09-21T15:33:53.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1600000000000001e-06 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118919+0000", 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4859999999999997e-06 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118923+0000", 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3390000000000003e-06 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.262 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86425 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122231+0000", 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.999723, 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014817350000000001, 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86425 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.263 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122231+0000", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122231+0000", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122231+0000", 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122233+0000", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1569999999999998e-06 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122239+0000", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5430000000000001e-06 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122243+0000", 2024-09-21T15:33:53.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6090000000000003e-06 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86071 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099376+0000", 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022578501000000001, 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024380898000000002, 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.266 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86071 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099376+0000", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099376+0000", 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099376+0000", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099378+0000", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.351e-06 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099385+0000", 2024-09-21T15:33:53.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8809999999999998e-06 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099390+0000", 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7790000000000003e-06 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86106 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099992+0000", 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021962492, 2024-09-21T15:33:53.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.023796787999999999, 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86106 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099992+0000", 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099992+0000", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099992+0000", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099999+0000", 2024-09-21T15:33:53.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3730000000000004e-06 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100005+0000", 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1369999999999999e-06 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100009+0000", 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3499999999999999e-06 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86138 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100636+0000", 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021318462999999999, 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02318338, 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86138 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100636+0000", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100636+0000", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100636+0000", 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100638+0000", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4229999999999999e-06 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100646+0000", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2400000000000007e-06 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100651+0000", 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7489999999999998e-06 2024-09-21T15:33:53.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86170 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.102238+0000", 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.019716712000000001, 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021610800999999999, 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86170 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102238+0000", 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102238+0000", 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102237+0000", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102240+0000", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3360000000000002e-06 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102246+0000", 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6590000000000001e-06 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102251+0000", 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2540000000000003e-06 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86330 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114749+0000", 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0072056819999999997, 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0091595229999999993, 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86330 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114749+0000", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114749+0000", 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114748+0000", 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114751+0000", 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7530000000000002e-06 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114758+0000", 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4970000000000004e-06 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114763+0000", 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7940000000000002e-06 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86362 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.282 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118286+0000", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0036679529999999998, 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0056784740000000002, 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86362 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118286+0000", 2024-09-21T15:33:53.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118286+0000", 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118286+0000", 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118294+0000", 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7239999999999992e-06 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118299+0000", 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6579999999999999e-06 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118304+0000", 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7129999999999996e-06 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.285 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86394 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118927+0000", 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0030278000000000002, 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050985639999999999, 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86394 2024-09-21T15:33:53.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118927+0000", 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118927+0000", 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118926+0000", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118928+0000", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.013e-06 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118935+0000", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2879999999999998e-06 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118939+0000", 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3830000000000002e-06 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86426 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122247+0000", 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9997072, 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00182833, 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86426 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122247+0000", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122247+0000", 2024-09-21T15:33:53.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122247+0000", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122249+0000", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1720000000000001e-06 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122255+0000", 2024-09-21T15:33:53.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5849999999999999e-06 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122259+0000", 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.498e-06 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86073 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099394+0000", 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022560362, 2024-09-21T15:33:53.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024715668, 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86073 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099394+0000", 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099394+0000", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099394+0000", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099396+0000", 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4200000000000001e-06 2024-09-21T15:33:53.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099402+0000", 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0070000000000003e-06 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099407+0000", 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6909999999999996e-06 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86331 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114768+0000", 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0071868319999999998, 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093711660000000002, 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86331 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114768+0000", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114768+0000", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114767+0000", 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114769+0000", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2850000000000001e-06 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114775+0000", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6500000000000001e-06 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114779+0000", 2024-09-21T15:33:53.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.386e-06 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86363 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118320+0000", 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0036342000000000002, 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058484510000000002, 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86363 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118320+0000", 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118320+0000", 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118320+0000", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118322+0000", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6680000000000001e-06 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118329+0000", 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0449999999999997e-06 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118334+0000", 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1429999999999997e-06 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86395 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118943+0000", 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0030115960000000001, 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0052538860000000001, 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.302 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86395 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118943+0000", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118943+0000", 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118943+0000", 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118945+0000", 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0049999999999999e-06 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118950+0000", 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2580000000000003e-06 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118954+0000", 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4580000000000003e-06 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86427 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.305 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122264+0000", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9996905, 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001963266, 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86427 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122264+0000", 2024-09-21T15:33:53.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122264+0000", 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122263+0000", 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122266+0000", 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0839999999999999e-06 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122271+0000", 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4399999999999996e-06 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122275+0000", 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3329999999999998e-06 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86075 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099411+0000", 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022543193, 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024856428, 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86075 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099411+0000", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099411+0000", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099411+0000", 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099413+0000", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.278e-06 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099420+0000", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9959999999999996e-06 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099425+0000", 2024-09-21T15:33:53.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9300000000000002e-06 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86108 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100031+0000", 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021922943, 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024300121000000001, 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86108 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100031+0000", 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100031+0000", 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100031+0000", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100033+0000", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1299999999999999e-06 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100039+0000", 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5929999999999996e-06 2024-09-21T15:33:53.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100043+0000", 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5199999999999999e-06 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86140 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100671+0000", 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021282946000000001, 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0237179, 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.315 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86140 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100671+0000", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100671+0000", 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100671+0000", 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100673+0000", 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1449999999999998e-06 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100679+0000", 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9880000000000001e-06 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100685+0000", 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4829999999999999e-06 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86332 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.318 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114783+0000", 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0071711819999999999, 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096672300000000006, 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86332 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114783+0000", 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114783+0000", 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114783+0000", 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114785+0000", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114790+0000", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3770000000000004e-06 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114795+0000", 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4139999999999996e-06 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.321 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86364 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118338+0000", 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0036162249999999998, 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0061497720000000004, 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86364 2024-09-21T15:33:53.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118338+0000", 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118338+0000", 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118338+0000", 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118340+0000", 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.216e-06 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118346+0000", 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5609999999999998e-06 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118350+0000", 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4259999999999996e-06 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86396 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118960+0000", 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.002994816, 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0055575609999999999, 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.325 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86396 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118960+0000", 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118960+0000", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118959+0000", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118962+0000", 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6539999999999999e-06 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118968+0000", 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9950000000000002e-06 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118972+0000", 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6809999999999997e-06 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86428 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.328 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122279+0000", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9996753, 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0022690240000000001, 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86428 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122279+0000", 2024-09-21T15:33:53.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122279+0000", 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122279+0000", 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122281+0000", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2649999999999999e-06 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122287+0000", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6350000000000003e-06 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122291+0000", 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4460000000000003e-06 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.331 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86077 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099429+0000", 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022525061999999998, 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025148288000000001, 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86077 2024-09-21T15:33:53.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099429+0000", 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099429+0000", 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099429+0000", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099431+0000", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2539999999999999e-06 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099438+0000", 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2620000000000004e-06 2024-09-21T15:33:53.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099442+0000", 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86173 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.102304+0000", 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.019650877000000001, 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022308369000000002, 2024-09-21T15:33:53.335 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86173 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102304+0000", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102304+0000", 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102303+0000", 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102305+0000", 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.249e-06 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102311+0000", 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3619999999999997e-06 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.102316+0000", 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0459999999999996e-06 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86333 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.338 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114799+0000", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0071559290000000001, 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098700609999999994, 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86333 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114799+0000", 2024-09-21T15:33:53.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114799+0000", 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114798+0000", 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114800+0000", 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9800000000000001e-06 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114806+0000", 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6869999999999999e-06 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114810+0000", 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5680000000000001e-06 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86365 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118354+0000", 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0036006010000000001, 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063747980000000001, 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86365 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.342 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118354+0000", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118354+0000", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118353+0000", 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118356+0000", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2850000000000001e-06 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118361+0000", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.378e-06 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118366+0000", 2024-09-21T15:33:53.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8670000000000002e-06 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86397 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118977+0000", 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0029775819999999999, 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058083700000000002, 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86397 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118977+0000", 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118977+0000", 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118976+0000", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118979+0000", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4839999999999998e-06 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118984+0000", 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7010000000000001e-06 2024-09-21T15:33:53.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118989+0000", 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5870000000000003e-06 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86429 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122295+0000", 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9996595, 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0025373980000000002, 2024-09-21T15:33:53.348 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86429 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122295+0000", 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122295+0000", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122295+0000", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122297+0000", 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0420000000000001e-06 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122302+0000", 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3929999999999999e-06 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122307+0000", 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.544e-06 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.351 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86110 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100087+0000", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021867199, 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024776019, 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86110 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100087+0000", 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100087+0000", 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100087+0000", 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100089+0000", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4600000000000002e-06 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100095+0000", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7529999999999998e-06 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100101+0000", 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.614e-06 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.354 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86142 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100707+0000", 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021247915999999999, 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02418735, 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86142 2024-09-21T15:33:53.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100707+0000", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100707+0000", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100706+0000", 2024-09-21T15:33:53.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100708+0000", 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2479999999999999e-06 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100714+0000", 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0000000000000002e-06 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100719+0000", 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.702e-06 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86334 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114814+0000", 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0071400300000000003, 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010108199, 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.358 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86334 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114814+0000", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114814+0000", 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114814+0000", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114816+0000", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2110000000000001e-06 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114822+0000", 2024-09-21T15:33:53.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5430000000000001e-06 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114826+0000", 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4050000000000002e-06 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86366 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118370+0000", 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0035845780000000002, 2024-09-21T15:33:53.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0066645109999999997, 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86366 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118370+0000", 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118370+0000", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118370+0000", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118372+0000", 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1050000000000002e-06 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118377+0000", 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.502e-06 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118382+0000", 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3919999999999996e-06 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.364 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86398 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119002+0000", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0029521040000000001, 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0060922060000000002, 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86398 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119002+0000", 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119002+0000", 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119002+0000", 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119004+0000", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119010+0000", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9869999999999996e-06 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119016+0000", 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3380000000000004e-06 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.367 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86430 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122331+0000", 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9996238, 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0028229399999999999, 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86430 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.368 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122331+0000", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122331+0000", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122330+0000", 2024-09-21T15:33:53.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122333+0000", 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.587e-06 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122339+0000", 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.19e-06 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122344+0000", 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8389999999999997e-06 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86079 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099469+0000", 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022485249999999998, 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025725606000000002, 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86079 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099469+0000", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099469+0000", 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099469+0000", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099472+0000", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8159999999999998e-06 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099480+0000", 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7430000000000008e-06 2024-09-21T15:33:53.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099485+0000", 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8049999999999997e-06 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86335 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114830+0000", 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.007124259, 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010393842, 2024-09-21T15:33:53.374 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86335 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114830+0000", 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114830+0000", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114830+0000", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114832+0000", 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2350000000000002e-06 2024-09-21T15:33:53.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114837+0000", 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3859999999999998e-06 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114842+0000", 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4179999999999999e-06 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.377 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86367 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118385+0000", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0035691, 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068712210000000003, 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86367 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118385+0000", 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118385+0000", 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118385+0000", 2024-09-21T15:33:53.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118387+0000", 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3180000000000001e-06 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118393+0000", 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3970000000000002e-06 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118398+0000", 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6570000000000002e-06 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86399 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119020+0000", 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0029347309999999999, 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0062657620000000002, 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.383 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86399 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119020+0000", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119020+0000", 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119019+0000", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119026+0000", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000004e-06 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119032+0000", 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6790000000000002e-06 2024-09-21T15:33:53.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119037+0000", 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5549999999999996e-06 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86431 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122348+0000", 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9996061, 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0029678399999999998, 2024-09-21T15:33:53.386 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86431 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122348+0000", 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122348+0000", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122348+0000", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122350+0000", 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4099999999999998e-06 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122356+0000", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6570000000000002e-06 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122362+0000", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2589999999999997e-06 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86112 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100120+0000", 2024-09-21T15:33:53.389 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021834453, 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025247200000000001, 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86112 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100120+0000", 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100120+0000", 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100120+0000", 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100122+0000", 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1840000000000002e-06 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100127+0000", 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5820000000000001e-06 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100132+0000", 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6920000000000001e-06 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86144 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100768+0000", 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021186561999999999, 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.024661523000000001, 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86144 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.393 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100768+0000", 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100768+0000", 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100767+0000", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100770+0000", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5550000000000001e-06 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100777+0000", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9979999999999997e-06 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100782+0000", 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.899e-06 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86336 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114846+0000", 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0071087240000000003, 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010647143, 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.396 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86336 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114846+0000", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114846+0000", 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114845+0000", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114847+0000", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114853+0000", 2024-09-21T15:33:53.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3759999999999999e-06 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114857+0000", 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2830000000000003e-06 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86368 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118402+0000", 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0035521860000000001, 2024-09-21T15:33:53.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0071431050000000003, 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86368 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118402+0000", 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118402+0000", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118402+0000", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118404+0000", 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1969999999999999e-06 2024-09-21T15:33:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118410+0000", 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6500000000000001e-06 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118414+0000", 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.437e-06 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86400 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119041+0000", 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0029138480000000001, 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065347069999999998, 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86400 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119041+0000", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119041+0000", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119040+0000", 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119042+0000", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0040000000000002e-06 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119048+0000", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119052+0000", 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4100000000000001e-06 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.405 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86432 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122368+0000", 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9995871, 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0032383949999999998, 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.406 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86432 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122368+0000", 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122368+0000", 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122367+0000", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122369+0000", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3259999999999998e-06 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122375+0000", 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.524e-06 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122382+0000", 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.793e-06 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86081 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099510+0000", 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022444827000000001, 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026126276, 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.409 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86081 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099510+0000", 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099510+0000", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099509+0000", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099512+0000", 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4200000000000001e-06 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099518+0000", 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099523+0000", 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7140000000000001e-06 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86337 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.412 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114862+0000", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.007092409, 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010816951999999999, 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86337 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114862+0000", 2024-09-21T15:33:53.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114862+0000", 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114862+0000", 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114864+0000", 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1909999999999999e-06 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114870+0000", 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8549999999999998e-06 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114875+0000", 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6949999999999997e-06 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.415 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86369 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118431+0000", 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0035232050000000002, 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0073079399999999997, 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86369 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118431+0000", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118431+0000", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118431+0000", 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118434+0000", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7719999999999999e-06 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118441+0000", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6290000000000001e-06 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118446+0000", 2024-09-21T15:33:53.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9930000000000003e-06 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86401 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119056+0000", 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0028988320000000001, 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0067418599999999997, 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86401 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119056+0000", 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119056+0000", 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119055+0000", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119057+0000", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9740000000000001e-06 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119063+0000", 2024-09-21T15:33:53.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4670000000000003e-06 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119067+0000", 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3749999999999996e-06 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86433 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122386+0000", 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9995685, 2024-09-21T15:33:53.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034745700000000002, 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86433 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122386+0000", 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122386+0000", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122385+0000", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122388+0000", 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5069999999999999e-06 2024-09-21T15:33:53.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122395+0000", 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8630000000000002e-06 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122399+0000", 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5809999999999998e-06 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86114 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100154+0000", 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021800452000000001, 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025749448000000001, 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86114 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100154+0000", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100154+0000", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100154+0000", 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100156+0000", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.125e-06 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100162+0000", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8880000000000002e-06 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100166+0000", 2024-09-21T15:33:53.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2910000000000001e-06 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86146 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100805+0000", 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021149903000000001, 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025132192000000001, 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.429 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86146 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100805+0000", 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100805+0000", 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100804+0000", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100806+0000", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2330000000000001e-06 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100813+0000", 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2469999999999997e-06 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100819+0000", 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2999999999999998e-06 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86338 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114880+0000", 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0070748310000000002, 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085668999999999, 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.432 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86338 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114880+0000", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114880+0000", 2024-09-21T15:33:53.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114879+0000", 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114886+0000", 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6819999999999997e-06 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114892+0000", 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.942e-06 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114896+0000", 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5850000000000001e-06 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86370 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.435 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118451+0000", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0035039369999999999, 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075451670000000002, 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86370 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118451+0000", 2024-09-21T15:33:53.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118451+0000", 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118450+0000", 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118452+0000", 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4480000000000001e-06 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118458+0000", 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5140000000000001e-06 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118462+0000", 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4900000000000002e-06 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.438 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86402 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119071+0000", 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0028835649999999998, 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0069651050000000001, 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86402 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.439 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119071+0000", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119071+0000", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119071+0000", 2024-09-21T15:33:53.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119073+0000", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0590000000000001e-06 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119078+0000", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5210000000000002e-06 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119083+0000", 2024-09-21T15:33:53.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7829999999999998e-06 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86434 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122403+0000", 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9995513, 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.003693684, 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.442 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86434 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122403+0000", 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122403+0000", 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122403+0000", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122405+0000", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0660000000000002e-06 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122411+0000", 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9669999999999998e-06 2024-09-21T15:33:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122415+0000", 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.651e-06 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86083 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099553+0000", 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022401896000000001, 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026609081999999999, 2024-09-21T15:33:53.445 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86083 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099553+0000", 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099553+0000", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099552+0000", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099555+0000", 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.79e-06 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099561+0000", 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5069999999999997e-06 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099567+0000", 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8880000000000002e-06 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86339 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.448 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114900+0000", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0070542290000000004, 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011323723000000001, 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86339 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114900+0000", 2024-09-21T15:33:53.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114900+0000", 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114900+0000", 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114902+0000", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9929999999999998e-06 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114907+0000", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5040000000000002e-06 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114912+0000", 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4299999999999999e-06 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.451 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86371 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118467+0000", 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0034875309999999999, 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077888599999999999, 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86371 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.452 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118467+0000", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118467+0000", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118467+0000", 2024-09-21T15:33:53.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118469+0000", 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9999999999999999e-06 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118474+0000", 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4559999999999999e-06 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118478+0000", 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3850000000000004e-06 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86403 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119105+0000", 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0028492439999999999, 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0071809869999999998, 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86403 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119105+0000", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119105+0000", 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119105+0000", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119107+0000", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7259999999999998e-06 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119114+0000", 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.19e-06 2024-09-21T15:33:53.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119119+0000", 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8860000000000003e-06 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86435 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122419+0000", 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9995351, 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0038958439999999999, 2024-09-21T15:33:53.458 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86435 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122419+0000", 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122419+0000", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122419+0000", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122421+0000", 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2730000000000001e-06 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122427+0000", 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5629999999999999e-06 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122431+0000", 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4150000000000001e-06 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86084 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.461 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099572+0000", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022382695000000001, 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026772820999999999, 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86084 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099572+0000", 2024-09-21T15:33:53.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099572+0000", 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099571+0000", 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099574+0000", 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5380000000000001e-06 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099580+0000", 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0279999999999997e-06 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099586+0000", 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9689999999999999e-06 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86116 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100187+0000", 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021767880999999999, 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026189917, 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86116 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100187+0000", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100187+0000", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100186+0000", 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100189+0000", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4040000000000002e-06 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100194+0000", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6810000000000003e-06 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100199+0000", 2024-09-21T15:33:53.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2889999999999999e-06 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86148 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100858+0000", 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021096835000000001, 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.025552750999999999, 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86148 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100858+0000", 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100858+0000", 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100857+0000", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100860+0000", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7319999999999998e-06 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100866+0000", 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.19e-06 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100871+0000", 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5689999999999998e-06 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86340 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114916+0000", 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00703872, 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011553081999999999, 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86340 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114916+0000", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114916+0000", 2024-09-21T15:33:53.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114915+0000", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114917+0000", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.181e-06 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114924+0000", 2024-09-21T15:33:53.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2709999999999998e-06 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114928+0000", 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4769999999999997e-06 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86372 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118482+0000", 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0034722799999999999, 2024-09-21T15:33:53.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080443189999999994, 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86372 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118482+0000", 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118482+0000", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118482+0000", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118484+0000", 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.097e-06 2024-09-21T15:33:53.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118490+0000", 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5100000000000004e-06 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118495+0000", 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8450000000000002e-06 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86404 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119123+0000", 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0028319199999999999, 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0074637360000000003, 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86404 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119123+0000", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119123+0000", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119122+0000", 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119124+0000", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.266e-06 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119130+0000", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4659999999999998e-06 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119134+0000", 2024-09-21T15:33:53.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6600000000000003e-06 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86436 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122440+0000", 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9995141, 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0041909770000000002, 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86436 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122440+0000", 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122440+0000", 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122440+0000", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122443+0000", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.537e-06 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122449+0000", 2024-09-21T15:33:53.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.139e-06 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122453+0000", 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6630000000000001e-06 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86341 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114932+0000", 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0070224149999999997, 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011732377, 2024-09-21T15:33:53.484 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86341 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114932+0000", 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114932+0000", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114932+0000", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114934+0000", 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.052e-06 2024-09-21T15:33:53.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114940+0000", 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1589999999999998e-06 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114944+0000", 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5249999999999999e-06 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.487 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86373 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118499+0000", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0034550449999999999, 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0081937850000000003, 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86373 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118499+0000", 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118499+0000", 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118499+0000", 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118501+0000", 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.402e-06 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118507+0000", 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6350000000000003e-06 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118511+0000", 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.403e-06 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86405 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119139+0000", 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0028159130000000002, 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0075891450000000003, 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.491 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86405 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119139+0000", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119139+0000", 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119138+0000", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119140+0000", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1349999999999999e-06 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119146+0000", 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2009999999999996e-06 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119151+0000", 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5290000000000002e-06 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86437 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122457+0000", 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994969, 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0043277230000000003, 2024-09-21T15:33:53.494 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86437 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122457+0000", 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122457+0000", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122457+0000", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122459+0000", 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0779999999999998e-06 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122464+0000", 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3140000000000003e-06 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122470+0000", 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1449999999999999e-06 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86086 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.497 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099609+0000", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022344950999999998, 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027237727999999999, 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86086 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099609+0000", 2024-09-21T15:33:53.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099609+0000", 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099609+0000", 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099612+0000", 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.396e-06 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099618+0000", 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.246e-06 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099623+0000", 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3689999999999998e-06 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86118 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100221+0000", 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021733382999999998, 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026685034, 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86118 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100221+0000", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100221+0000", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100221+0000", 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100223+0000", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4370000000000001e-06 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100230+0000", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6850000000000004e-06 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100234+0000", 2024-09-21T15:33:53.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7049999999999998e-06 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86150 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100892+0000", 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021062096999999998, 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026053449999999999, 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86150 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100892+0000", 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100892+0000", 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100892+0000", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100894+0000", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2579999999999998e-06 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100900+0000", 2024-09-21T15:33:53.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.003e-06 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100905+0000", 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7319999999999998e-06 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86342 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114949+0000", 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0070057460000000002, 2024-09-21T15:33:53.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012028596000000001, 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86342 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114949+0000", 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114949+0000", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114948+0000", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114951+0000", 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5129999999999999e-06 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114957+0000", 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7969999999999997e-06 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114961+0000", 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6550000000000003e-06 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86374 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118515+0000", 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00343919, 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084906600000000006, 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86374 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118515+0000", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118515+0000", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118515+0000", 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118517+0000", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9879999999999999e-06 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118522+0000", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5419999999999996e-06 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118527+0000", 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3900000000000003e-06 2024-09-21T15:33:53.513 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86406 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119156+0000", 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027983729999999998, 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078796449999999994, 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.514 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86406 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119156+0000", 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119156+0000", 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119156+0000", 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119158+0000", 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0339999999999999e-06 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119163+0000", 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5470000000000004e-06 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119169+0000", 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2079999999999999e-06 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86438 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122473+0000", 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994807, 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.004591464, 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86438 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122473+0000", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122473+0000", 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122473+0000", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122475+0000", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9690000000000001e-06 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122481+0000", 2024-09-21T15:33:53.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2219999999999999e-06 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122486+0000", 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9119999999999997e-06 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86343 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114966+0000", 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0069883219999999999, 2024-09-21T15:33:53.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012138925, 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86343 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114966+0000", 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114966+0000", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114966+0000", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114968+0000", 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.978e-06 2024-09-21T15:33:53.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114973+0000", 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4970000000000001e-06 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114978+0000", 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5510000000000001e-06 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86375 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118546+0000", 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0034080080000000001, 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0086148309999999999, 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86375 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118546+0000", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118546+0000", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118546+0000", 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118561+0000", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4956e-05 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118568+0000", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9670000000000004e-06 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118573+0000", 2024-09-21T15:33:53.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0429999999999998e-06 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86407 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119172+0000", 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027820549999999999, 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080502860000000002, 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.527 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86407 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119172+0000", 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119172+0000", 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119172+0000", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119174+0000", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1440000000000001e-06 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119179+0000", 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3650000000000003e-06 2024-09-21T15:33:53.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119184+0000", 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3819999999999997e-06 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86439 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122491+0000", 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994636, 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.004789944, 2024-09-21T15:33:53.530 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86439 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122491+0000", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122491+0000", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122490+0000", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122493+0000", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4210000000000002e-06 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122498+0000", 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6180000000000003e-06 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122503+0000", 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4399999999999998e-06 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86088 3.d 3:b5af49aa:::rbd_data.1191f95e298.0000000000000006:18 [read 7159808~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.533 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.099644+0000", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.022310276, 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027680716000000001, 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86088 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099644+0000", 2024-09-21T15:33:53.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099644+0000", 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099644+0000", 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099646+0000", 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.368e-06 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099652+0000", 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0909999999999998e-06 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.099657+0000", 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5410000000000002e-06 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.127254+0000", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027597494 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.127295+0000", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0862999999999998e-05 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.127328+0000", 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2500999999999999e-05 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86120 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100271+0000", 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021683139000000001, 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027092379, 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86120 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.538 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100271+0000", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100271+0000", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100271+0000", 2024-09-21T15:33:53.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100274+0000", 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9579999999999999e-06 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100280+0000", 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1890000000000004e-06 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100285+0000", 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.87e-06 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86152 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100940+0000", 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021014807999999999, 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026454502000000001, 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86152 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100940+0000", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100940+0000", 2024-09-21T15:33:53.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100939+0000", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100942+0000", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100949+0000", 2024-09-21T15:33:53.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2069999999999998e-06 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100954+0000", 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1739999999999999e-06 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86344 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114982+0000", 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0069720190000000003, 2024-09-21T15:33:53.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012440166000000001, 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86344 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114982+0000", 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114982+0000", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114982+0000", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114984+0000", 2024-09-21T15:33:53.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.261e-06 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114990+0000", 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.49e-06 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114994+0000", 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4240000000000003e-06 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86376 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118577+0000", 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0033770509999999998, 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0088896449999999998, 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86376 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118577+0000", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118577+0000", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118577+0000", 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118579+0000", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1409999999999999e-06 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118584+0000", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4349999999999996e-06 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118589+0000", 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3630000000000004e-06 2024-09-21T15:33:53.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86408 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119188+0000", 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027666940000000001, 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0083362660000000002, 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86408 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119188+0000", 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119188+0000", 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119187+0000", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119189+0000", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1969999999999999e-06 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119196+0000", 2024-09-21T15:33:53.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0669999999999997e-06 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119200+0000", 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.741e-06 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86440 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122507+0000", 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994478, 2024-09-21T15:33:53.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050785930000000002, 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86440 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122507+0000", 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122507+0000", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122506+0000", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122508+0000", 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.238e-06 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122514+0000", 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4659999999999998e-06 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122518+0000", 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3259999999999997e-06 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86345 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.557 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.114998+0000", 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00695635, 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012646806, 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86345 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114998+0000", 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114998+0000", 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.114998+0000", 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115000+0000", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1909999999999999e-06 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115005+0000", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4659999999999998e-06 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115010+0000", 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3209999999999998e-06 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.560 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86377 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118593+0000", 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0033617959999999998, 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0090854219999999992, 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.561 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86377 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118593+0000", 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118593+0000", 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118592+0000", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118594+0000", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.058e-06 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118600+0000", 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3730000000000001e-06 2024-09-21T15:33:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118604+0000", 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.454e-06 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86409 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119212+0000", 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027419689999999999, 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084968379999999996, 2024-09-21T15:33:53.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86409 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119212+0000", 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119212+0000", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119212+0000", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119215+0000", 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5569999999999998e-06 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119221+0000", 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119226+0000", 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8099999999999997e-06 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86441 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122522+0000", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994326, 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0052165969999999999, 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86441 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122522+0000", 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122522+0000", 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122522+0000", 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122524+0000", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4849999999999999e-06 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122530+0000", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7049999999999996e-06 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122534+0000", 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3710000000000002e-06 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.570 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86122 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100307+0000", 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021647046, 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027463988000000002, 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86122 2024-09-21T15:33:53.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100307+0000", 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100307+0000", 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100307+0000", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100309+0000", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4430000000000002e-06 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100315+0000", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7019999999999998e-06 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100319+0000", 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4299999999999999e-06 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86154 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100976+0000", 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020978635999999998, 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026826929999999999, 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86154 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100976+0000", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100976+0000", 2024-09-21T15:33:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100975+0000", 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100978+0000", 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4980000000000001e-06 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100984+0000", 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.083e-06 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100990+0000", 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8769999999999998e-06 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86346 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115013+0000", 2024-09-21T15:33:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0069409759999999997, 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012848249000000001, 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86346 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115013+0000", 2024-09-21T15:33:53.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115013+0000", 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115013+0000", 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115015+0000", 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2539999999999999e-06 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115021+0000", 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8420000000000001e-06 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115026+0000", 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8450000000000002e-06 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86378 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118608+0000", 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0033463550000000001, 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0093127590000000003, 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86378 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118608+0000", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118608+0000", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118608+0000", 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118610+0000", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2469999999999998e-06 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118615+0000", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2959999999999998e-06 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118620+0000", 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9250000000000003e-06 2024-09-21T15:33:53.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.584 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.584 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.585 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86410 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.585 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119230+0000", 2024-09-21T15:33:53.585 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027240659999999998, 2024-09-21T15:33:53.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0087507620000000005, 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86410 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119230+0000", 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119230+0000", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119230+0000", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119232+0000", 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2170000000000001e-06 2024-09-21T15:33:53.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119238+0000", 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.434e-06 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119244+0000", 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1360000000000002e-06 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.588 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86442 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122538+0000", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994164, 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0054898380000000004, 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86442 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122538+0000", 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122538+0000", 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122538+0000", 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122540+0000", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.091e-06 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122545+0000", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2830000000000001e-06 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122549+0000", 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.391e-06 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.591 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86347 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115030+0000", 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0069241880000000004, 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013154545, 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86347 2024-09-21T15:33:53.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115030+0000", 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115030+0000", 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115030+0000", 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115032+0000", 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2529999999999999e-06 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115037+0000", 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4070000000000001e-06 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115042+0000", 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4909999999999999e-06 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86379 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118624+0000", 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.003330024, 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096183290000000001, 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86379 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118624+0000", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118624+0000", 2024-09-21T15:33:53.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118624+0000", 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118626+0000", 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2649999999999999e-06 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118632+0000", 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.665e-06 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118637+0000", 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4589999999999997e-06 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.598 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86411 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119248+0000", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0027068280000000001, 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0090558870000000003, 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86411 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119248+0000", 2024-09-21T15:33:53.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119248+0000", 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119247+0000", 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119254+0000", 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3810000000000004e-06 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119259+0000", 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6520000000000003e-06 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119264+0000", 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6809999999999997e-06 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86443 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122553+0000", 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9994016, 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058012430000000002, 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86443 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122553+0000", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122553+0000", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122553+0000", 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122555+0000", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.103e-06 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122560+0000", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3410000000000002e-06 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122565+0000", 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.775e-06 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86124 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100340+0000", 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021614901999999998, 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028044990999999998, 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86124 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100340+0000", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100340+0000", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100339+0000", 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100341+0000", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3870000000000002e-06 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100347+0000", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5589999999999996e-06 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100351+0000", 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2209999999999999e-06 2024-09-21T15:33:53.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86156 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101017+0000", 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020937317, 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027399201000000001, 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.608 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86156 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101017+0000", 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101017+0000", 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101017+0000", 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101019+0000", 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4710000000000001e-06 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101025+0000", 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0430000000000004e-06 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101030+0000", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.622e-06 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86348 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115046+0000", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0069088860000000004, 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013399588, 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.611 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86348 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115046+0000", 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115046+0000", 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115045+0000", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115047+0000", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.058e-06 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115053+0000", 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4519999999999996e-06 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115057+0000", 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3560000000000003e-06 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86380 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118641+0000", 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.003313046, 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098334849999999994, 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.614 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86380 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118641+0000", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118641+0000", 2024-09-21T15:33:53.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118641+0000", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118643+0000", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0779999999999998e-06 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118649+0000", 2024-09-21T15:33:53.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1920000000000002e-06 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118654+0000", 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.42e-06 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86412 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119268+0000", 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0026865439999999999, 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092426030000000003, 2024-09-21T15:33:53.617 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86412 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119268+0000", 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119268+0000", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119267+0000", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119270+0000", 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2000000000000001e-06 2024-09-21T15:33:53.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119275+0000", 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.524e-06 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119280+0000", 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5229999999999997e-06 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86444 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122569+0000", 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9993854, 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006000839, 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86444 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122569+0000", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122569+0000", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122569+0000", 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122571+0000", 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1840000000000002e-06 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122577+0000", 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.575e-06 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122581+0000", 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4499999999999997e-06 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86157 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101034+0000", 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020920213999999999, 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.027594535999999999, 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.624 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86157 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101034+0000", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101034+0000", 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101034+0000", 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101036+0000", 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2680000000000001e-06 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101042+0000", 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1059999999999996e-06 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101047+0000", 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5240000000000002e-06 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86349 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115061+0000", 2024-09-21T15:33:53.627 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0068934529999999999, 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013628748, 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86349 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115061+0000", 2024-09-21T15:33:53.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115061+0000", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115061+0000", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115063+0000", 2024-09-21T15:33:53.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1160000000000002e-06 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115068+0000", 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4160000000000003e-06 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115072+0000", 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3830000000000002e-06 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.630 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86381 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118669+0000", 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032850140000000002, 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010061703999999999, 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86381 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.631 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118669+0000", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118669+0000", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118669+0000", 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118671+0000", 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4260000000000002e-06 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118677+0000", 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118682+0000", 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0939999999999998e-06 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86413 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119285+0000", 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0026695180000000001, 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0094775239999999993, 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86413 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119285+0000", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119285+0000", 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119285+0000", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119287+0000", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0449999999999999e-06 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119293+0000", 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3260000000000001e-06 2024-09-21T15:33:53.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119297+0000", 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4340000000000002e-06 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86445 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122585+0000", 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9993696, 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0062070679999999996, 2024-09-21T15:33:53.637 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86445 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122585+0000", 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122585+0000", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122584+0000", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122586+0000", 2024-09-21T15:33:53.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0660000000000002e-06 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122592+0000", 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3820000000000003e-06 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122596+0000", 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3869999999999997e-06 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86126 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100373+0000", 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021581703000000001, 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028451712000000001, 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86126 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100373+0000", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100373+0000", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100372+0000", 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100375+0000", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3929999999999998e-06 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100380+0000", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5589999999999996e-06 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100385+0000", 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5020000000000003e-06 2024-09-21T15:33:53.643 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86350 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115086+0000", 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0068686559999999999, 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013768157, 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.644 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86350 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115086+0000", 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115086+0000", 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115085+0000", 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115092+0000", 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7170000000000002e-06 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115098+0000", 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1249999999999998e-06 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115103+0000", 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2360000000000004e-06 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86382 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118686+0000", 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032681870000000001, 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010229518, 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.647 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86382 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118686+0000", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118686+0000", 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118686+0000", 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118688+0000", 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4739999999999999e-06 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118697+0000", 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3629999999999994e-06 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118701+0000", 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7870000000000001e-06 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86414 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.650 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119317+0000", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0026376239999999999, 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0096565049999999993, 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86414 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119317+0000", 2024-09-21T15:33:53.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119317+0000", 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119316+0000", 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119319+0000", 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5179999999999999e-06 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119325+0000", 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9739999999999999e-06 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119329+0000", 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4730000000000002e-06 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.653 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86446 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122600+0000", 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9993544, 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0064311530000000002, 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86446 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.654 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122600+0000", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122600+0000", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122600+0000", 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122602+0000", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2409999999999998e-06 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122607+0000", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.553e-06 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122612+0000", 2024-09-21T15:33:53.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5170000000000001e-06 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86159 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101079+0000", 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020875496, 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028002916999999999, 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.657 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86159 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101079+0000", 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101079+0000", 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101078+0000", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101081+0000", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7829999999999999e-06 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101088+0000", 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.742e-06 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101094+0000", 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7270000000000004e-06 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86351 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115108+0000", 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0068463969999999997, 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01400289, 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.660 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86351 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115108+0000", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115108+0000", 2024-09-21T15:33:53.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115108+0000", 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115110+0000", 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7650000000000002e-06 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115116+0000", 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8030000000000002e-06 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115122+0000", 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.395e-06 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86383 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118706+0000", 2024-09-21T15:33:53.663 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032488310000000002, 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010436486, 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86383 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118706+0000", 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118706+0000", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118705+0000", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118708+0000", 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4490000000000002e-06 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118713+0000", 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5849999999999999e-06 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118719+0000", 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8799999999999996e-06 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.666 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86415 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119333+0000", 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0026211099999999998, 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.009842188, 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86415 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119333+0000", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119333+0000", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119333+0000", 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119335+0000", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.435e-06 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119341+0000", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1020000000000002e-06 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119347+0000", 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1080000000000001e-06 2024-09-21T15:33:53.669 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86447 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122616+0000", 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9993382, 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.006619504, 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86447 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122616+0000", 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122616+0000", 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122616+0000", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122618+0000", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.384e-06 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122625+0000", 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6379999999999998e-06 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122629+0000", 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6190000000000002e-06 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86128 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.100417+0000", 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.021537851, 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028878073000000001, 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.673 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86128 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100417+0000", 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100417+0000", 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100416+0000", 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100419+0000", 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7659999999999999e-06 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100426+0000", 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7329999999999997e-06 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.100431+0000", 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9219999999999996e-06 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86352 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.676 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115126+0000", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0068289270000000003, 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014229388000000001, 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86352 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115126+0000", 2024-09-21T15:33:53.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115126+0000", 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115125+0000", 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115127+0000", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.869e-06 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115132+0000", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3410000000000002e-06 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115137+0000", 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3769999999999998e-06 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.679 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86384 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118723+0000", 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032316200000000002, 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010676849, 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86384 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.680 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118723+0000", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118723+0000", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118722+0000", 2024-09-21T15:33:53.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118725+0000", 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1959999999999998e-06 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118730+0000", 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4419999999999997e-06 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118734+0000", 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.313e-06 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86416 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119351+0000", 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0026035590000000001, 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010079882, 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.683 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86416 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119351+0000", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119351+0000", 2024-09-21T15:33:53.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119350+0000", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119353+0000", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2469999999999998e-06 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119358+0000", 2024-09-21T15:33:53.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.6860000000000003e-06 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119363+0000", 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86448 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122633+0000", 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.999321, 2024-09-21T15:33:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068263569999999999, 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86448 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122633+0000", 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122633+0000", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122633+0000", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122646+0000", 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3281e-05 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122654+0000", 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5909999999999998e-06 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122659+0000", 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0170000000000004e-06 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.689 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86161 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101126+0000", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020828177, 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028364248000000002, 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86161 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101126+0000", 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101126+0000", 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101126+0000", 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101128+0000", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.418e-06 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101136+0000", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0620000000000001e-06 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101141+0000", 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5700000000000003e-06 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86353 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115259+0000", 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.006695777, 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014260518999999999, 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86353 2024-09-21T15:33:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115259+0000", 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115259+0000", 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115140+0000", 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998813 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115262+0000", 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000121694 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115270+0000", 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5860000000000005e-06 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115276+0000", 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.015e-06 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86385 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118738+0000", 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032160819999999999, 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010815452, 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86385 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118738+0000", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118738+0000", 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118738+0000", 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118740+0000", 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1390000000000002e-06 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118745+0000", 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3510000000000001e-06 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118750+0000", 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.189e-06 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86417 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.699 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119367+0000", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00258772, 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010245651999999999, 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86417 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119367+0000", 2024-09-21T15:33:53.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119367+0000", 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119366+0000", 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119368+0000", 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0190000000000001e-06 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119374+0000", 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2630000000000003e-06 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119378+0000", 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3660000000000002e-06 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.702 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86449 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122663+0000", 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9992914, 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0070077580000000002, 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86449 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122663+0000", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122663+0000", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122662+0000", 2024-09-21T15:33:53.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122665+0000", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0669999999999999e-06 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122670+0000", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3430000000000003e-06 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122674+0000", 2024-09-21T15:33:53.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.267e-06 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86162 3.d 3.5592f5ad (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.101374+0000", 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.020580742999999999, 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028359926000000001, 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86162 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101374+0000", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101374+0000", 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101145+0000", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997716 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101377+0000", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00023210799999999999 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101388+0000", 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0579e-05 2024-09-21T15:33:53.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.101394+0000", 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7070000000000003e-06 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86354 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115283+0000", 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0066717900000000004, 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014495609, 2024-09-21T15:33:53.709 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86354 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115283+0000", 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115283+0000", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115282+0000", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115285+0000", 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6180000000000002e-06 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115291+0000", 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9150000000000001e-06 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115295+0000", 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7520000000000004e-06 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86386 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.712 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118754+0000", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0032008919999999999, 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011057724, 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86386 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118754+0000", 2024-09-21T15:33:53.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118754+0000", 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118753+0000", 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118755+0000", 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9410000000000001e-06 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118761+0000", 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3049999999999998e-06 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118766+0000", 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5399999999999997e-06 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86418 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119539+0000", 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0024159390000000002, 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010301045, 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86418 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119539+0000", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119539+0000", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119382+0000", 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998431 2024-09-21T15:33:53.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119541+0000", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000159918 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119549+0000", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6820000000000003e-06 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119555+0000", 2024-09-21T15:33:53.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7049999999999996e-06 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86450 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122699+0000", 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.9992552, 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0071710699999999999, 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86450 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122699+0000", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122699+0000", 2024-09-21T15:33:53.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122699+0000", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122702+0000", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7829999999999999e-06 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122708+0000", 2024-09-21T15:33:53.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.348e-06 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122712+0000", 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4560000000000002e-06 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86355 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.115299+0000", 2024-09-21T15:33:53.722 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0066549879999999997, 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014601380000000001, 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86355 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115299+0000", 2024-09-21T15:33:53.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115299+0000", 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115299+0000", 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115301+0000", 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2919999999999998e-06 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115307+0000", 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.434e-06 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.115311+0000", 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5820000000000003e-06 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.725 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86387 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.118786+0000", 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0031687320000000001, 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011171028, 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86387 2024-09-21T15:33:53.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118786+0000", 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118786+0000", 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118785+0000", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118798+0000", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2558000000000001e-05 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118805+0000", 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8260000000000004e-06 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.118810+0000", 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0010000000000001e-06 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86419 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.119560+0000", 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.0023946459999999998, 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.010455007000000001, 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.729 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86419 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119560+0000", 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119560+0000", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119559+0000", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119562+0000", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.142e-06 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119567+0000", 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.553e-06 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.119572+0000", 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3919999999999996e-06 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:86451 3.e 3.5657286e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:53.122716+0000", 2024-09-21T15:33:53.732 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 4294967295.999238, 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0073585509999999996, 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "queued for pg", 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 86451 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122716+0000", 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122716+0000", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122716+0000", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122718+0000", 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1210000000000001e-06 2024-09-21T15:33:53.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122724+0000", 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3600000000000004e-06 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:53.122728+0000", 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2300000000000002e-06 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:53.735 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T15:33:53.736 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 182 2024-09-21T15:33:53.736 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:53.738 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:53.738 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:53.737+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:33:53.738 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 71, 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:33:51.888904+0000", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:33:20.030364+0000", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5804032, 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "nodeep-scrub", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:33:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "luminous", 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:33:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:33:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:33:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:33:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "70", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 26, 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 70, 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:33:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:33:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:33:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:33:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 56, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:33:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.75054931640625, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:33:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 57, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.8086395263671875, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:33:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.784 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:33:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:33:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:33:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:33:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:33:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:33:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 53, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.71441650390625, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 68, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:33:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:33:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:33:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [ 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.10", 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: 7, 2024-09-21T15:33:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: 5 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:33:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [], 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:33:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:33:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:33:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:53.814 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:53.815 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:33:51.888904+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:33:20.030364+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 0.75054931640625, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 0.8086395263671875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 0.71441650390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.10', 'osds': [7, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:33:53.815 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T15:33:53.860 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:53.861+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:53.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:33:53.938 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:53.935+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:53.988 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:53.989 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:54.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:54.053 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:54.053+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:54.141 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:54.139+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:54.200 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47133 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9123840~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1368170339999999, 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091148090000000001, 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.201 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47133 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063912+0000", 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063911+0000", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063916+0000", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6110000000000004e-06 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063928+0000", 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1948e-05 2024-09-21T15:33:54.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063937+0000", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2930000000000006e-06 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.154776+0000", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090839016999999994 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.154905+0000", 2024-09-21T15:33:54.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012873299999999999 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155060+0000", 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000155205 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47134 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9127936~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1367834069999994, 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091431745999999994, 2024-09-21T15:33:54.205 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47134 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063946+0000", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063945+0000", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063949+0000", 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.391e-06 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063961+0000", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1888e-05 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063972+0000", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1027e-05 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155106+0000", 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091133567999999998 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155229+0000", 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012347899999999999 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155378+0000", 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014811300000000001 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47135 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9132032~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.209 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1367478910000006, 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091711168999999995, 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47135 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063981+0000", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063985+0000", 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6940000000000003e-06 2024-09-21T15:33:54.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.063997+0000", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2152e-05 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064007+0000", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1670000000000005e-06 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155422+0000", 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091415095000000002 2024-09-21T15:33:54.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155548+0000", 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012659300000000001 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155692+0000", 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014420500000000001 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.213 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47136 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9136128~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1367138259999994, 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091991677999999993, 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47136 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064015+0000", 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064019+0000", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3579999999999996e-06 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064031+0000", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1941e-05 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064040+0000", 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.0599999999999997e-06 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155738+0000", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091698084999999999 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.155862+0000", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012419300000000001 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156007+0000", 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014476300000000001 2024-09-21T15:33:54.217 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47137 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9140224~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1366026169999994, 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092194095000000004, 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.218 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47137 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064127+0000", 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064048+0000", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999208 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064133+0000", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5834000000000005e-05 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064152+0000", 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8505e-05 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.064165+0000", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2853e-05 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156048+0000", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091882894000000007 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156168+0000", 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120517 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.156321+0000", 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015246500000000001 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47140 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9414656~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1343329690000008, 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091029811000000002, 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.222 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47140 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066396+0000", 2024-09-21T15:33:54.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066395+0000", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066400+0000", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8180000000000003e-06 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066412+0000", 2024-09-21T15:33:54.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1888999999999999e-05 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066422+0000", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0355e-05 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157158+0000", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090735646000000003 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157285+0000", 2024-09-21T15:33:54.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012709599999999999 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157426+0000", 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000140906 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47141 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9418752~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1342991219999998, 2024-09-21T15:33:54.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091333279000000003, 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47141 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066430+0000", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066429+0000", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066434+0000", 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0000000000000004e-06 2024-09-21T15:33:54.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066447+0000", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2128e-05 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066457+0000", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9820000000000003e-06 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157486+0000", 2024-09-21T15:33:54.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091029652000000003 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157617+0000", 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013117000000000001 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157763+0000", 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000146012 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47142 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9422848~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1342635350000005, 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091614438000000006, 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47142 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066466+0000", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066465+0000", 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066470+0000", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6909999999999996e-06 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066481+0000", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.182e-05 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066492+0000", 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0169999999999999e-05 2024-09-21T15:33:54.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157814+0000", 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091322540999999993 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.157937+0000", 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012265 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158080+0000", 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014326200000000001 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47143 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9426944~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1342213490000006, 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092001538999999993, 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.235 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47143 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066508+0000", 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066507+0000", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066512+0000", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4160000000000003e-06 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066525+0000", 2024-09-21T15:33:54.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2383e-05 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.066535+0000", 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0247e-05 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158180+0000", 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091645236000000005 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158334+0000", 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015351199999999999 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.158509+0000", 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017578299999999999 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47152 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8547328~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:54.239 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1303307769999993, 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091150810999999998, 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47152 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070398+0000", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070401+0000", 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.542e-06 2024-09-21T15:33:54.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070407+0000", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0870000000000003e-06 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070412+0000", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1189999999999996e-06 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161282+0000", 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090869968999999995 2024-09-21T15:33:54.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161405+0000", 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012354999999999999 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161549+0000", 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000143969 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.243 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47153 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8551424~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1303128069999993, 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091466130000000007, 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47153 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070416+0000", 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070419+0000", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6139999999999999e-06 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070428+0000", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4700000000000008e-06 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.070433+0000", 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2399999999999998e-06 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161601+0000", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091167699000000005 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161735+0000", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013359299999999999 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.161883+0000", 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014793900000000001 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.247 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47163 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9116672~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1262986910000006, 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090996833999999999, 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47163 2024-09-21T15:33:54.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074431+0000", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074430+0000", 2024-09-21T15:33:54.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074433+0000", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8329999999999998e-06 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074439+0000", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0870000000000003e-06 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074444+0000", 2024-09-21T15:33:54.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9690000000000002e-06 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165147+0000", 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090702609000000003 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165283+0000", 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013657 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165427+0000", 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014419199999999999 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47164 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9120768~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1262806800000007, 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091301265000000006, 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47164 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074449+0000", 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074448+0000", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074451+0000", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5959999999999998e-06 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074458+0000", 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5390000000000002e-06 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.074463+0000", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1420000000000001e-06 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165477+0000", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091014034999999993 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165604+0000", 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012615700000000001 2024-09-21T15:33:54.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.165750+0000", 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014622499999999999 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47175 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9427968~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1222027600000004, 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091182622000000005, 2024-09-21T15:33:54.256 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47175 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078526+0000", 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078529+0000", 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.802e-06 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078536+0000", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2370000000000003e-06 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.078541+0000", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4090000000000002e-06 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169371+0000", 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090829441999999996 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169528+0000", 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015705800000000001 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.169709+0000", 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018110399999999999 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47204 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9461760~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.260 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1178514629999992, 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090974816, 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47204 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082878+0000", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082877+0000", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082881+0000", 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1219999999999997e-06 2024-09-21T15:33:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082895+0000", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3621999999999999e-05 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082903+0000", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5229999999999995e-06 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173578+0000", 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090674169999999998 2024-09-21T15:33:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173710+0000", 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013205000000000001 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173853+0000", 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014296900000000001 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.264 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47207 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9465856~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1177416539999996, 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091198736000000002, 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47207 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:54.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082988+0000", 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082910+0000", 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999228 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.082994+0000", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3819000000000005e-05 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.083009+0000", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4645e-05 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.083020+0000", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1024e-05 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.173904+0000", 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090884548999999995 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.174027+0000", 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000122583 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.174186+0000", 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015947199999999999 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.268 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47229 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10035200~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1139264749999995, 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091206161999999993, 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47229 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.269 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086803+0000", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086802+0000", 2024-09-21T15:33:54.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086807+0000", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7659999999999998e-06 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086818+0000", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0997e-05 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.086828+0000", 2024-09-21T15:33:54.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0729e-05 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.177737+0000", 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090908364000000005 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.177868+0000", 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000131283 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.178009+0000", 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014094300000000001 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47240 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10342400~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1098773929999997, 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090925040999999998, 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47240 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090852+0000", 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090851+0000", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090855+0000", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2220000000000004e-06 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090868+0000", 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3301e-05 2024-09-21T15:33:54.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090878+0000", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3640000000000005e-06 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181499+0000", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090620777999999999 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181632+0000", 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000133332 2024-09-21T15:33:54.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181777+0000", 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014491999999999999 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47241 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 10346496~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1098434229999992, 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091328454000000003, 2024-09-21T15:33:54.277 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47241 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090886+0000", 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090885+0000", 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090889+0000", 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2320000000000003e-06 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090900+0000", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0796e-05 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.090909+0000", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4530000000000002e-06 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.181862+0000", 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090953645999999999 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.182018+0000", 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015544500000000001 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.182214+0000", 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019647600000000001 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.281 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:47251 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9733120~1024] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 9.1060235439999992, 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090993061, 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 47251 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:54.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094706+0000", 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094705+0000", 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:54.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094708+0000", 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8890000000000002e-06 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094717+0000", 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5310000000000001e-06 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.094722+0000", 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3929999999999999e-06 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185367+0000", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090644481999999998 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185521+0000", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154661 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:45.185699+0000", 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000177534 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.285 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.286 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:54.286 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:54.286 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:54.287 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:54.285+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:54.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:54.388 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:54.387+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:54.431 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:54.431 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:54.431 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:54.432 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:54.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:54.492 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:54.491+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:54.590 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:54.589+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:54.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:54.619 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:54.619 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:54.619 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:54.619 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:54.619 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:54.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:54.690 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:54.689+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:54.791 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:54.789+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:54.800 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:54.801 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:54.824 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:54.824 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:54.825 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:54.825 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:54.846 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:54.846 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:54.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:54.878 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:54.892 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:54.891+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:54.904 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:54.904 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:54.913 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:33:54.930 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:54.949 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:54.950 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:54.969 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:54.969 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:54.983 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:54.983 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:54.993 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:54.993+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:54.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:55.094 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:55.091+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:55.157 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:55.157 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:55.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:55.195 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:55.193+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:55.296 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:55.293+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:55.332 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:55.332 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:55.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:55.397 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:55.395+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:55.468 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:55.468 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:55.481 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:33:55.524 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:55.523+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:55.598 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:55.595+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:55.653 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:55.654 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:55.668 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:33:55.712 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:55.711+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:55.801 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:55.799+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:55.836 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:55.836 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:55.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:33:55.940 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:55.939+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:56.003 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:55.999+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:56.034 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.035 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:33:56.104 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:56.101+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:56.205 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:56.205+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:56.221 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.221 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:33:56.306 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:56.305+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.379 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.396 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:33:56.441 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:56.441+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:56.507 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:56.505+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:56.542 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.542 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.555 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:33:56.608 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:56.607+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:56.709 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:56.709+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:56.716 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.716 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.718 INFO:teuthology.orchestra.run.smithi107.stdout:2878 trunc from 0xa3f21f1 to 0x648a128 2024-09-21T15:33:56.732 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:33:56.762 INFO:teuthology.orchestra.run.smithi107.stdout:2879 read 0x5dc0c69 thru 0x5dd041e (0xf7b6 bytes) 2024-09-21T15:33:56.763 INFO:teuthology.orchestra.run.smithi107.stdout:2880 punch from 0x15fcc4b to 0x160746c, (0xa821 bytes) 2024-09-21T15:33:56.763 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:56.763 INFO:teuthology.orchestra.run.smithi107.stdout:2882 punch from 0x52b63ab to 0x52b65f0, (0x245 bytes) 2024-09-21T15:33:56.763 INFO:teuthology.orchestra.run.smithi107.stdout:2883 read 0x55f81e7 thru 0x55fc379 (0x4193 bytes) 2024-09-21T15:33:56.763 INFO:teuthology.orchestra.run.smithi107.stdout:2886 writesame 0xa3e446 thru 0xa479b2 data_size 0x3(0x956d bytes) 2024-09-21T15:33:56.791 INFO:teuthology.orchestra.run.smithi107.stdout:2887 punch from 0x434a58d to 0x434c56d, (0x1fe0 bytes) 2024-09-21T15:33:56.795 INFO:teuthology.orchestra.run.smithi107.stdout:2888 trunc from 0x648a128 to 0x44aafa8 2024-09-21T15:33:56.810 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:56.807+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:56.836 INFO:teuthology.orchestra.run.smithi107.stdout:2889 read 0x117ef30 thru 0x1185fa1 (0x7072 bytes) 2024-09-21T15:33:56.836 INFO:teuthology.orchestra.run.smithi107.stdout:2891 read 0x3ce1698 thru 0x3ce7882 (0x61eb bytes) 2024-09-21T15:33:56.855 INFO:teuthology.orchestra.run.smithi107.stdout:2892 writesame 0x88a9925 thru 0x88acfeb data_size 0x1(0x36c7 bytes) 2024-09-21T15:33:56.855 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:56.856 INFO:teuthology.orchestra.run.smithi107.stdout:2894 read 0x83ada36 thru 0x83bc913 (0xeede bytes) 2024-09-21T15:33:56.856 INFO:teuthology.orchestra.run.smithi107.stdout:2896 read 0x843ca1a thru 0x8449a5f (0xd046 bytes) 2024-09-21T15:33:56.856 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:56.856 INFO:teuthology.orchestra.run.smithi107.stdout:2898 write 0x258d178 thru 0x258ddef (0xc78 bytes) 2024-09-21T15:33:56.872 INFO:teuthology.orchestra.run.smithi107.stdout:2899 trunc from 0x88acfec to 0xccde890 2024-09-21T15:33:56.885 INFO:teuthology.orchestra.run.smithi107.stdout:2900 write 0x4a34b0 thru 0x4acfed (0x9b3e bytes) 2024-09-21T15:33:56.911 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:56.907+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:56.921 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:56.921 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:56.923 INFO:teuthology.orchestra.run.smithi107.stdout:2901 read 0xa1367c7 thru 0xa13b95b (0x5195 bytes) 2024-09-21T15:33:56.923 INFO:teuthology.orchestra.run.smithi107.stdout:2902 writesame 0x6318414 thru 0x63258ee data_size 0x1(0xd4db bytes) 2024-09-21T15:33:56.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:33:56.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:33:56.971 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:56.971 INFO:teuthology.orchestra.run.smithi107.stdout:2904 write 0x1e93deb thru 0x1e9c7bd (0x89d3 bytes) 2024-09-21T15:33:56.981 INFO:teuthology.orchestra.run.smithi107.stdout:2905 punch from 0x4b79b35 to 0x4b8056f, (0x6a3a bytes) 2024-09-21T15:33:56.981 INFO:teuthology.orchestra.run.smithi107.stdout:2906 write 0x7fe4a95 thru 0x7fef39d (0xa909 bytes) 2024-09-21T15:33:56.987 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:33:56.988 INFO:teuthology.orchestra.run.smithi107.stdout:2907 punch from 0xcb0a567 to 0xcb11ebe, (0x7957 bytes) 2024-09-21T15:33:56.988 INFO:teuthology.orchestra.run.smithi107.stdout:2909 read 0xb9520a7 thru 0xb95efe4 (0xcf3e bytes) 2024-09-21T15:33:56.989 INFO:teuthology.orchestra.run.smithi107.stdout:2911 read 0x5755ee0 thru 0x575d105 (0x7226 bytes) 2024-09-21T15:33:56.989 INFO:teuthology.orchestra.run.smithi107.stdout:2912 read 0x37e06b6 thru 0x37e74e6 (0x6e31 bytes) 2024-09-21T15:33:56.989 INFO:teuthology.orchestra.run.smithi107.stdout:2913 write 0x174cc5a thru 0x17515b7 (0x495e bytes) 2024-09-21T15:33:57.012 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:57.009+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:57.016 INFO:teuthology.orchestra.run.smithi107.stdout:2914 trunc from 0xccde890 to 0x16814e3 2024-09-21T15:33:57.057 INFO:teuthology.orchestra.run.smithi107.stdout:2915 write 0x3756109 thru 0x375ea78 (0x8970 bytes) 2024-09-21T15:33:57.064 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.067 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:57.067 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:57.080 INFO:teuthology.orchestra.run.smithi107.stdout:2918 writesame 0x7a34cc8 thru 0x7a402a5 data_size 0x1(0xb5de bytes) 2024-09-21T15:33:57.080 INFO:teuthology.orchestra.run.smithi107.stdout:2919 write 0x352c7cf thru 0x3536369 (0x9b9b bytes) 2024-09-21T15:33:57.083 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:33:57.086 INFO:teuthology.orchestra.run.smithi107.stdout:2920 read 0xcb14fd thru 0xcb1b33 (0x637 bytes) 2024-09-21T15:33:57.086 INFO:teuthology.orchestra.run.smithi107.stdout:2924 read 0x76652a1 thru 0x766ed6e (0x9ace bytes) 2024-09-21T15:33:57.102 INFO:teuthology.orchestra.run.smithi107.stdout:2925 writesame 0xde9fc1e thru 0xdeae972 data_size 0x1(0xed55 bytes) 2024-09-21T15:33:57.102 INFO:teuthology.orchestra.run.smithi107.stdout:2928 read 0x6c3eb35 thru 0x6c45378 (0x6844 bytes) 2024-09-21T15:33:57.107 INFO:teuthology.orchestra.run.smithi107.stdout:2929 trunc from 0xdeae973 to 0xa1fbb81 2024-09-21T15:33:57.112 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:57.111+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:57.121 INFO:teuthology.orchestra.run.smithi107.stdout:2932 write 0xac9b5a8 thru 0xac9e61a (0x3073 bytes) 2024-09-21T15:33:57.126 INFO:teuthology.orchestra.run.smithi107.stdout:2933 write 0x14a61c5 thru 0x14a8330 (0x216c bytes) 2024-09-21T15:33:57.129 INFO:teuthology.orchestra.run.smithi107.stdout:2935 punch from 0x4fe628f to 0x4fee7d6, (0x8547 bytes) 2024-09-21T15:33:57.142 INFO:teuthology.orchestra.run.smithi107.stdout:2936 writesame 0xe66f122 thru 0xe67e165 data_size 0x2(0xf044 bytes) 2024-09-21T15:33:57.142 INFO:teuthology.orchestra.run.smithi107.stdout:2938 read 0x8131de9 thru 0x813c0f1 (0xa309 bytes) 2024-09-21T15:33:57.142 INFO:teuthology.orchestra.run.smithi107.stdout:2939 write 0x90ed69d thru 0x90f8632 (0xaf96 bytes) 2024-09-21T15:33:57.151 INFO:teuthology.orchestra.run.smithi107.stdout:2941 read 0x8f52a23 thru 0x8f56101 (0x36df bytes) 2024-09-21T15:33:57.155 INFO:teuthology.orchestra.run.smithi107.stdout:2942 trunc from 0xe67e166 to 0xba75ac3 2024-09-21T15:33:57.157 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:57.158 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:57.161 INFO:teuthology.orchestra.run.smithi107.stdout:2945 writesame 0x4a40c92 thru 0x4a40de0 data_size 0x1(0x14f bytes) 2024-09-21T15:33:57.165 INFO:teuthology.orchestra.run.smithi107.stdout:2946 write 0xc60741 thru 0xc62ef9 (0x27b9 bytes) 2024-09-21T15:33:57.166 INFO:teuthology.orchestra.run.smithi107.stdout:2949 punch from 0x10a0857 to 0x10a6e2f, (0x65d8 bytes) 2024-09-21T15:33:57.167 INFO:teuthology.orchestra.run.smithi107.stdout:2950 read 0x6a2fe63 thru 0x6a310ca (0x1268 bytes) 2024-09-21T15:33:57.167 INFO:teuthology.orchestra.run.smithi107.stdout:2951 writesame 0x4557350 thru 0x4561894 data_size 0x9(0xa545 bytes) 2024-09-21T15:33:57.174 INFO:teuthology.orchestra.run.smithi107.stdout:2957 read 0x52c93ad thru 0x52d62ab (0xceff bytes) 2024-09-21T15:33:57.179 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:57.180 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:57.185 INFO:teuthology.orchestra.run.smithi107.stdout:2958 write 0xe4040dc thru 0xe40d0aa (0x8fcf bytes) 2024-09-21T15:33:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.192 INFO:teuthology.orchestra.run.smithi107.stdout:2961 write 0x672a879 thru 0x672a970 (0xf8 bytes) 2024-09-21T15:33:57.196 INFO:teuthology.orchestra.run.smithi107.stdout:2962 write 0xcdd04bf thru 0xcdd0f0d (0xa4f bytes) 2024-09-21T15:33:57.198 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:57.200 INFO:teuthology.orchestra.run.smithi107.stdout:2963 read 0xccf15f3 thru 0xccf77ce (0x61dc bytes) 2024-09-21T15:33:57.201 INFO:teuthology.orchestra.run.smithi107.stdout:2967 writesame 0x3c74a30 thru 0x3c761a0 data_size 0x1(0x1771 bytes) 2024-09-21T15:33:57.207 INFO:teuthology.orchestra.run.smithi107.stdout:2968 punch from 0xa35491f to 0xa3588d7, (0x3fb8 bytes) 2024-09-21T15:33:57.213 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:57.213+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:57.216 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:57.217 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:57.224 INFO:teuthology.orchestra.run.smithi107.stdout:2969 trunc from 0xe40d0ab to 0x2259e41 2024-09-21T15:33:57.242 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:57.243 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:57.255 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:57.255 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:2971 writesame 0x5c93759 thru 0x5ca2b0e data_size 0x22(0xf3b6 bytes) 2024-09-21T15:33:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:2974 write 0x11514f8 thru 0x115c263 (0xad6c bytes) 2024-09-21T15:33:57.264 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:57.264 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:57.272 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:33:57.276 INFO:teuthology.orchestra.run.smithi107.stdout:2975 read 0x144c65 thru 0x14cb02 (0x7e9e bytes) 2024-09-21T15:33:57.276 INFO:teuthology.orchestra.run.smithi107.stdout:2977 punch from 0x808e1a to 0x80c7d5, (0x39bb bytes) 2024-09-21T15:33:57.283 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:57.283 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:57.284 INFO:teuthology.orchestra.run.smithi107.stdout:2978 trunc from 0x5ca2b0f to 0xb43210c 2024-09-21T15:33:57.293 INFO:teuthology.orchestra.run.smithi107.stdout:2981 trunc from 0xb43210c to 0xcff342a 2024-09-21T15:33:57.299 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.299 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.299 INFO:teuthology.orchestra.run.smithi107.stdout:2984 write 0x3dc4dfb thru 0x3dd0b56 (0xbd5c bytes) 2024-09-21T15:33:57.306 INFO:teuthology.orchestra.run.smithi107.stdout:2985 write 0x849d45b thru 0x84a51d9 (0x7d7f bytes) 2024-09-21T15:33:57.307 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:57.307 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:57.312 INFO:teuthology.orchestra.run.smithi107.stdout:2986 trunc from 0xcff342a to 0xd022616 2024-09-21T15:33:57.314 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:57.311+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:2987 writesame 0xebb2f0e thru 0xebb7302 data_size 0x3(0x43f5 bytes) 2024-09-21T15:33:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:2989 read 0x27950e3 thru 0x279c9a0 (0x78be bytes) 2024-09-21T15:33:57.328 INFO:teuthology.orchestra.run.smithi107.stdout:2990 read 0x4dcfe7f thru 0x4dd1bbe (0x1d40 bytes) 2024-09-21T15:33:57.328 INFO:teuthology.orchestra.run.smithi107.stdout:2991 read 0x68a8c2d thru 0x68b86da (0xfaae bytes) 2024-09-21T15:33:57.328 INFO:teuthology.orchestra.run.smithi107.stdout:2992 writesame 0x87eb616 thru 0x87f84ea data_size 0x1(0xced5 bytes) 2024-09-21T15:33:57.345 INFO:teuthology.orchestra.run.smithi107.stdout:2995 writesame 0x3db9d00 thru 0x3dc1b09 data_size 0x2(0x7e0a bytes) 2024-09-21T15:33:57.355 INFO:teuthology.orchestra.run.smithi107.stdout:2996 write 0x1344eca thru 0x134f99b (0xaad2 bytes) 2024-09-21T15:33:57.366 INFO:teuthology.orchestra.run.smithi107.stdout:2998 write 0x5bf130a thru 0x5bf6834 (0x552b bytes) 2024-09-21T15:33:57.375 INFO:teuthology.orchestra.run.smithi107.stdout:2999 trunc from 0xebb7303 to 0xbd86311 2024-09-21T15:33:57.380 INFO:teuthology.orchestra.run.smithi107.stdout:3000 read 0x92acf1b thru 0x92bb589 (0xe66f bytes) 2024-09-21T15:33:57.381 INFO:teuthology.orchestra.run.smithi107.stdout:3001 writesame 0x1ed2f2c thru 0x1edb033 data_size 0x4(0x8108 bytes) 2024-09-21T15:33:57.385 INFO:teuthology.orchestra.run.smithi107.stdout:3003 write 0xba77fef thru 0xba7aa9f (0x2ab1 bytes) 2024-09-21T15:33:57.390 INFO:teuthology.orchestra.run.smithi107.stdout:3004 writesame 0x7e18008 thru 0x7e25c9c data_size 0x7(0xdc95 bytes) 2024-09-21T15:33:57.401 INFO:teuthology.orchestra.run.smithi107.stdout:3005 read 0x46dede5 thru 0x46e2a95 (0x3cb1 bytes) 2024-09-21T15:33:57.401 INFO:teuthology.orchestra.run.smithi107.stdout:3006 writesame 0x261da06 thru 0x26265d1 data_size 0x2e(0x8bcc bytes) 2024-09-21T15:33:57.404 INFO:teuthology.orchestra.run.smithi107.stdout:3007 read 0xaf057fd thru 0xaf09114 (0x3918 bytes) 2024-09-21T15:33:57.404 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.404 INFO:teuthology.orchestra.run.smithi107.stdout:3009 write 0x740e9f3 thru 0x741430a (0x5918 bytes) 2024-09-21T15:33:57.411 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.411 INFO:teuthology.orchestra.run.smithi107.stdout:3012 write 0x3a2ed9b thru 0x3a3d003 (0xe269 bytes) 2024-09-21T15:33:57.415 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:57.411+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:57.423 INFO:teuthology.orchestra.run.smithi107.stdout:3013 trunc from 0xbd86311 to 0x73188dd 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:57.457 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 80.667129579000004, 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0092454589999999993, 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787384+0000", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787377+0000", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999924 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787399+0000", 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2073999999999999e-05 2024-09-21T15:33:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787435+0000", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6235999999999997e-05 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787464+0000", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8836999999999999e-05 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787594+0000", 2024-09-21T15:33:57.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000130436 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.787621+0000", 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6718999999999999e-05 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794150+0000", 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065284100000000001 2024-09-21T15:33:57.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794471+0000", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00032148399999999997 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 2", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.794858+0000", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038683400000000002 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796034+0000", 2024-09-21T15:33:57.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0011760010000000001 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796491+0000", 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045759899999999999 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796574+0000", 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2745999999999997e-05 2024-09-21T15:33:57.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:36.796630+0000", 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5747e-05 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-21T15:33:57.464 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 78.660518373000002, 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098767239999999999, 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4211", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/1736246127", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793996+0000", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.793993+0000", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794008+0000", 2024-09-21T15:33:57.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5429000000000001e-05 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794043+0000", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4659999999999997e-05 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794063+0000", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9930000000000001e-05 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794216+0000", 2024-09-21T15:33:57.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015333600000000001 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.794241+0000", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4448999999999999e-05 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801128+0000", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0068876709999999997 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801274+0000", 2024-09-21T15:33:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145499 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 1", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.801492+0000", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00021865799999999999 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803021+0000", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001528918 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803745+0000", 2024-09-21T15:33:57.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00072329100000000004 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803815+0000", 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0690000000000003e-05 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:38.803872+0000", 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7139000000000003e-05 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:612 3.1d 3:ba6b47db:::rbd_data.11911fd826fd.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 65.085690654000004, 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012892621999999999, 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 612 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.471 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368823+0000", 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368820+0000", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368836+0000", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5943e-05 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368868+0000", 2024-09-21T15:33:57.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2276000000000001e-05 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368884+0000", 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5517000000000001e-05 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.368993+0000", 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010875200000000001 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.369312+0000", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000319275 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.371713+0000", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0024011309999999999 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381615+0000", 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0099017099999999993 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381626+0000", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1788e-05 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381668+0000", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.1118999999999997e-05 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:52.381716+0000", 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8494e-05 2024-09-21T15:33:57.476 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1234 3.c 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.783876661000001, 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093927381000000004, 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.477 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1234 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670637+0000", 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.670634+0000", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683920+0000", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013285853 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683966+0000", 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.57e-05 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.683987+0000", 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1077000000000002e-05 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684096+0000", 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000109109 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.684389+0000", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029284800000000001 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.712516+0000", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.028127276 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764284+0000", 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.051768243999999998 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764325+0000", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0386000000000003e-05 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764435+0000", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011024299999999999 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764565+0000", 2024-09-21T15:33:57.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012984499999999999 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1234 3.c e27/21 ondisk, result = 0)", 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.705060007, 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014837123000000001, 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749454+0000", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749452+0000", 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749466+0000", 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4163e-05 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749503+0000", 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7509999999999998e-05 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.749529+0000", 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.599e-05 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764258+0000", 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.01472921 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764274+0000", 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.554e-05 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.764291+0000", 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.715e-05 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1276 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.674706993999997, 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.065105479999999993, 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.487 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1276 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779807+0000", 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.779804+0000", 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787571+0000", 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077664659999999996 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787616+0000", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5704000000000002e-05 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787643+0000", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6953000000000001e-05 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787756+0000", 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112719 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787985+0000", 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000228848 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.790296+0000", 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0023113949999999999 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844730+0000", 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054433639999999998 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844747+0000", 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7537000000000002e-05 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844844+0000", 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.6608999999999996e-05 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844913+0000", 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8589000000000003e-05 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1279 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.666835298999999, 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.072065074000000007, 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.493 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1279 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787679+0000", 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.787676+0000", 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798732+0000", 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011056062 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798793+0000", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0795999999999999e-05 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798819+0000", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6176000000000001e-05 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826042+0000", 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.02722339 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826335+0000", 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029314500000000003 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.827082+0000", 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00074613999999999995 2024-09-21T15:33:57.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859414+0000", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.032331956000000002 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859474+0000", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0089e-05 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859561+0000", 2024-09-21T15:33:57.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7583999999999994e-05 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.859744+0000", 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018257899999999999 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.499 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1280 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.655655850999999, 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.071089756000000004, 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1280 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:57.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798858+0000", 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.798856+0000", 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813843+0000", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014986845 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813893+0000", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0188999999999997e-05 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.813920+0000", 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6789999999999999e-05 2024-09-21T15:33:57.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845017+0000", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.031097192999999999 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845375+0000", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035837100000000002 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.846078+0000", 2024-09-21T15:33:57.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00070301700000000001 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.863202+0000", 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017123413000000001 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869794+0000", 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0065922999999999997 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869876+0000", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1856999999999999e-05 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.869948+0000", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2244999999999997e-05 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1278 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'9, mlcod=27'6)", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:57.505 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.650423484999997, 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.040911251000000003, 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804091+0000", 2024-09-21T15:33:57.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.804087+0000", 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816199+0000", 2024-09-21T15:33:57.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012111957 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816243+0000", 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3976e-05 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.816265+0000", 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2243e-05 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.825987+0000", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0097221019999999998 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.826053+0000", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5345999999999994e-05 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844930+0000", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.018877771000000002 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.845002+0000", 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1372999999999996e-05 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop_reply(client.4497.0:1276 3.1d e27/21 ondisk, result = 0)", 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.631938204000001, 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022348920000000001, 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:57.510 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822576+0000", 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822573+0000", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822588+0000", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5189e-05 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822629+0000", 2024-09-21T15:33:57.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0890000000000003e-05 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.822651+0000", 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2643999999999999e-05 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844701+0000", 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.022049348999999999 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844720+0000", 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8683000000000001e-05 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.844925+0000", 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020525799999999999 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.514 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1319 3.1d 3:bb988149:::rbd_data.1191bb6b225c.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.504784702000002, 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014252015999999999, 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1319 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949729+0000", 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949726+0000", 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999962 2024-09-21T15:33:57.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949744+0000", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8196999999999999e-05 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949791+0000", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7626000000000001e-05 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949815+0000", 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3283000000000001e-05 2024-09-21T15:33:57.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949943+0000", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128042 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950227+0000", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028434800000000002 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.950580+0000", 2024-09-21T15:33:57.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035305100000000001 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963837+0000", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013257277 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963854+0000", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6623999999999999e-05 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963942+0000", 2024-09-21T15:33:57.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7755000000000003e-05 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.963981+0000", 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9629999999999998e-05 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1320 3.1d 3:be4f6cf6:::rbd_data.1191bb6b225c.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.50462838, 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.019624246000000001, 2024-09-21T15:33:57.520 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1320 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949886+0000", 2024-09-21T15:33:57.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949884+0000", 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:57.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949895+0000", 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0665999999999999e-05 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949913+0000", 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.842e-05 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.949930+0000", 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6779999999999999e-05 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958373+0000", 2024-09-21T15:33:57.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0084437039999999998 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958520+0000", 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014615400000000001 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958704+0000", 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018461000000000001 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964062+0000", 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053576530000000004 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969384+0000", 2024-09-21T15:33:57.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0053220849999999998 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969456+0000", 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1793999999999997e-05 2024-09-21T15:33:57.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969510+0000", 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4132999999999998e-05 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.531 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:1322 3.1d 3:ba3c4b56:::rbd_data.1191bb6b225c.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.498946154999999, 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.014332273, 2024-09-21T15:33:57.532 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 1322 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955568+0000", 2024-09-21T15:33:57.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955566+0000", 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:57.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955575+0000", 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8470000000000002e-06 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955590+0000", 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5104e-05 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.955602+0000", 2024-09-21T15:33:57.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1707e-05 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964111+0000", 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0085093059999999995 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964298+0000", 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000187014 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.964523+0000", 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022467599999999999 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969617+0000", 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0050937120000000002 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969772+0000", 2024-09-21T15:33:57.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015528499999999999 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969841+0000", 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8799000000000003e-05 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969900+0000", 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9228000000000003e-05 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:1323 3.d e27/21 3:b6c3ed76:::rbd_data.1191bb6b225c.0000000000000005:head v 27'10, mlcod=27'8)", 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.497983335999997, 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.013098662, 2024-09-21T15:33:57.543 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956531+0000", 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956529+0000", 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956544+0000", 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4521e-05 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956564+0000", 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0307000000000001e-05 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.956576+0000", 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2078e-05 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958370+0000", 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0017934139999999999 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.958431+0000", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1414999999999994e-05 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969517+0000", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.011085563 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:53.969629+0000", 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011286 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7720 3.1c 3:3965ba96:::rbd_data.1191ebf845d7.0000000000000006:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.737138412999997, 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.021370271999999999, 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7720 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717376+0000", 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717373+0000", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717387+0000", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4049000000000001e-05 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717417+0000", 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9366999999999999e-05 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717431+0000", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4693e-05 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717586+0000", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154962 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.717715+0000", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012838899999999999 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.721117+0000", 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0034019139999999998 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738541+0000", 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017424193000000001 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738615+0000", 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4589e-05 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738692+0000", 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6406e-05 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:07.738746+0000", 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4106e-05 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:7895 3.1c 3:39947815:::rbd_data.1191ebf845d7.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 49.428113779, 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.038890656000000003, 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 7895 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026400+0000", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026398+0000", 2024-09-21T15:33:57.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026409+0000", 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0611000000000001e-05 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026429+0000", 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0562999999999999e-05 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026440+0000", 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0424e-05 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026538+0000", 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8214000000000005e-05 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.026614+0000", 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5746000000000003e-05 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.031200+0000", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0045860370000000003 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065203+0000", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.034003316999999998 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065214+0000", 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0841e-05 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065256+0000", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2385999999999997e-05 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:08.065291+0000", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4817999999999998e-05 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:11984 3.5 3:a3251f67:::rbd_data.11917b2c111e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.149048802000003, 2024-09-21T15:33:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0080643109999999994, 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 11984 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:57.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305465+0000", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305462+0000", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305479+0000", 2024-09-21T15:33:57.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7513e-05 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305522+0000", 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.261e-05 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305546+0000", 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4284e-05 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305706+0000", 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016029799999999999 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op must wait for map", 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.305730+0000", 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.3096e-05 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311587+0000", 2024-09-21T15:33:57.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0058573449999999999 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311753+0000", 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016627499999999999 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.311904+0000", 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000151182 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313358+0000", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0014533390000000001 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313392+0000", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3924000000000002e-05 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313474+0000", 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2028999999999997e-05 2024-09-21T15:33:57.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.313530+0000", 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5875999999999998e-05 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:12160 3.5 3:a6273991:::rbd_data.11917b2c111e.000000000000001b:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 40.818209305000003, 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.017738413000000001, 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.568 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 12160 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636305+0000", 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636303+0000", 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636315+0000", 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1864e-05 2024-09-21T15:33:57.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636344+0000", 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9306e-05 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636358+0000", 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3873000000000001e-05 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636419+0000", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1393e-05 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.636530+0000", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011098 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.638734+0000", 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.002203595 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653854+0000", 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015120352 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.653960+0000", 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010642299999999999 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654008+0000", 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7817e-05 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:16.654043+0000", 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4863999999999997e-05 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.574 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26861 3.1c 3:3bbf239b:::rbd_data.1191e6bca1cd.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.148360871000001, 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0087661500000000003, 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26861 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306153+0000", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.306150+0000", 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307085+0000", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000935209 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307123+0000", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8003999999999997e-05 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307141+0000", 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7842e-05 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307265+0000", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124389 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307530+0000", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026449399999999999 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.307916+0000", 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038618499999999999 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314315+0000", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0063993590000000003 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314679+0000", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036339 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314820+0000", 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014107400000000001 2024-09-21T15:33:57.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.314919+0000", 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9418000000000003e-05 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:26864 3.1c 3:3c39757f:::rbd_data.1191e6bca1cd.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:57.580 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 17.145066958000001, 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0077293780000000003, 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 26864 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309447+0000", 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.309444+0000", 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:57.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310084+0000", 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00063922000000000002 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310138+0000", 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.4676000000000003e-05 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310160+0000", 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1325999999999999e-05 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.310956+0000", 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00079680599999999995 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311180+0000", 2024-09-21T15:33:57.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00022337600000000001 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.311734+0000", 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055437499999999996 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.315365+0000", 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0036305460000000001 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.316842+0000", 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001476904 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317088+0000", 2024-09-21T15:33:57.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024673799999999998 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:40.317176+0000", 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8107000000000002e-05 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.589 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:57.589 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:57.589 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:57.592 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:33:57.595 INFO:teuthology.orchestra.run.smithi107.stdout:3014 write 0xd43338b thru 0xd43ea13 (0xb689 bytes) 2024-09-21T15:33:57.595 INFO:teuthology.orchestra.run.smithi107.stdout:3017 write 0x25978fa thru 0x2597af8 (0x1ff bytes) 2024-09-21T15:33:57.595 INFO:teuthology.orchestra.run.smithi107.stdout:3018 punch from 0x51c7ccf to 0x51c7fcc, (0x2fd bytes) 2024-09-21T15:33:57.595 INFO:teuthology.orchestra.run.smithi107.stdout:3020 writesame 0x5f0c469 thru 0x5f19d7f data_size 0x2d(0xd917 bytes) 2024-09-21T15:33:57.595 INFO:teuthology.orchestra.run.smithi107.stdout:3021 flatten 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3022 read 0x33dc01f thru 0x33de263 (0x2245 bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3024 write 0xe62ef2f thru 0xe63b587 (0xc659 bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3025 read 0x80e4be1 thru 0x80e8ac5 (0x3ee5 bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3027 write 0xb3fce33 thru 0xb404626 (0x77f4 bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3028 write 0x1c7c84b thru 0x1c7f377 (0x2b2d bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3030 write 0xe032c81 thru 0xe03b709 (0x8a89 bytes) 2024-09-21T15:33:57.596 INFO:teuthology.orchestra.run.smithi107.stdout:3031 punch from 0x8ed4e49 to 0x8ee441d, (0xf5d4 bytes) 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:3032 read 0xe53cd95 thru 0xe547296 (0xa502 bytes) 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:3037 read 0xaa85d11 thru 0xaa8d74d (0x7a3d bytes) 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:3038 write 0x2bbffeb thru 0x2bcf562 (0xf578 bytes) 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:3039 write 0x61e1a72 thru 0x61eaae5 (0x9074 bytes) 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.597 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:3042 read 0x9651f7d thru 0x9655ba4 (0x3c28 bytes) 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:3043 write 0xc7ae054 thru 0xc7af83b (0x17e8 bytes) 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:3046 writesame 0x1c80502 thru 0x1c8277c data_size 0xd(0x227b bytes) 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:3049 write 0x6040309 thru 0x604b6e9 (0xb3e1 bytes) 2024-09-21T15:33:57.598 INFO:teuthology.orchestra.run.smithi107.stdout:3051 write 0x4f0da44 thru 0x4f1468c (0x6c49 bytes) 2024-09-21T15:33:57.599 INFO:teuthology.orchestra.run.smithi107.stdout:3052 write 0xa3bc4db thru 0xa3c661c (0xa142 bytes) 2024-09-21T15:33:57.599 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:57.587+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:57.600 INFO:teuthology.orchestra.run.smithi107.stdout:3053 writesame 0xc621851 thru 0xc6271d8 data_size 0x3c(0x5988 bytes) 2024-09-21T15:33:57.600 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.600 INFO:teuthology.orchestra.run.smithi107.stdout:3056 write 0x80bfc5f thru 0x80c7f11 (0x82b3 bytes) 2024-09-21T15:33:57.602 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.602 INFO:teuthology.orchestra.run.smithi107.stdout:3058 read 0xc17ec07 thru 0xc181318 (0x2712 bytes) 2024-09-21T15:33:57.602 INFO:teuthology.orchestra.run.smithi107.stdout:3059 write 0x1a45f6b thru 0x1a4bb97 (0x5c2d bytes) 2024-09-21T15:33:57.605 INFO:teuthology.orchestra.run.smithi107.stdout:3060 read 0x6d15a37 thru 0x6d248d6 (0xeea0 bytes) 2024-09-21T15:33:57.612 INFO:teuthology.orchestra.run.smithi107.stdout:3061 trunc from 0xe63b588 to 0xc440e19 2024-09-21T15:33:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:3064 read 0x1aebe4f thru 0x1af0011 (0x41c3 bytes) 2024-09-21T15:33:57.649 INFO:teuthology.orchestra.run.smithi107.stdout:3065 write 0xde3a57f thru 0xde3f596 (0x5018 bytes) 2024-09-21T15:33:57.657 INFO:teuthology.orchestra.run.smithi107.stdout:3066 punch from 0x3918fba to 0x391be47, (0x2e8d bytes) 2024-09-21T15:33:57.657 INFO:teuthology.orchestra.run.smithi107.stdout:3067 read 0xc792754 thru 0xc7a0844 (0xe0f1 bytes) 2024-09-21T15:33:57.657 INFO:teuthology.orchestra.run.smithi107.stdout:3068 punch from 0xd4308d5 to 0xd43bebc, (0xb5e7 bytes) 2024-09-21T15:33:57.658 INFO:teuthology.orchestra.run.smithi107.stdout:3069 write 0xc3d502f thru 0xc3e49ab (0xf97d bytes) 2024-09-21T15:33:57.670 INFO:teuthology.orchestra.run.smithi107.stdout:3070 punch from 0xd2d6676 to 0xd2e3626, (0xcfb0 bytes) 2024-09-21T15:33:57.670 INFO:teuthology.orchestra.run.smithi107.stdout:3071 writesame 0x879ef30 thru 0x87a2980 data_size 0x1(0x3a51 bytes) 2024-09-21T15:33:57.681 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.681 INFO:teuthology.orchestra.run.smithi107.stdout:3073 read 0x5ba51a1 thru 0x5babd09 (0x6b69 bytes) 2024-09-21T15:33:57.682 INFO:teuthology.orchestra.run.smithi107.stdout:3074 writesame 0x7b7599c thru 0x7b7bbe0 data_size 0x1(0x6245 bytes) 2024-09-21T15:33:57.691 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:57.689+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:57.714 INFO:teuthology.orchestra.run.smithi107.stdout:3075 read 0x30fb401 thru 0x3100ecd (0x5acd bytes) 2024-09-21T15:33:57.715 INFO:teuthology.orchestra.run.smithi107.stdout:3077 write 0xbcd1aae thru 0xbcd8cb1 (0x7204 bytes) 2024-09-21T15:33:57.719 INFO:teuthology.orchestra.run.smithi107.stdout:3079 writesame 0xa5d96fc thru 0xa5e2119 data_size 0x8e(0x8a1e bytes) 2024-09-21T15:33:57.731 INFO:teuthology.orchestra.run.smithi107.stdout:3080 read 0x4d11ace thru 0x4d1bd51 (0xa284 bytes) 2024-09-21T15:33:57.731 INFO:teuthology.orchestra.run.smithi107.stdout:3081 write 0xe2a34c thru 0xe39a79 (0xf72e bytes) 2024-09-21T15:33:57.740 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:57.740 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:57.740 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:57.754 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:33:57.757 INFO:teuthology.orchestra.run.smithi107.stdout:3082 trunc from 0xde3f597 to 0x6b589c6 2024-09-21T15:33:57.792 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:57.791+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:57.793 INFO:teuthology.orchestra.run.smithi107.stdout:3083 read 0x11f8d21 thru 0x11ffec4 (0x71a4 bytes) 2024-09-21T15:33:57.794 INFO:teuthology.orchestra.run.smithi107.stdout:3084 write 0x293a136 thru 0x2947fd4 (0xde9f bytes) 2024-09-21T15:33:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:3085 write 0x24731c2 thru 0x2475440 (0x227f bytes) 2024-09-21T15:33:57.822 INFO:teuthology.orchestra.run.smithi107.stdout:3086 write 0xa169bf8 thru 0xa177a16 (0xde1f bytes) 2024-09-21T15:33:57.841 INFO:teuthology.orchestra.run.smithi107.stdout:3087 trunc from 0xa177a17 to 0xc5afc9a 2024-09-21T15:33:57.851 INFO:teuthology.orchestra.run.smithi107.stdout:3088 punch from 0x22f2772 to 0x22fed57, (0xc5e5 bytes) 2024-09-21T15:33:57.851 INFO:teuthology.orchestra.run.smithi107.stdout:3089 writesame 0x4217bf1 thru 0x421dc79 data_size 0x1(0x6089 bytes) 2024-09-21T15:33:57.869 INFO:teuthology.orchestra.run.smithi107.stdout:3091 read 0x28f1090 thru 0x28f4805 (0x3776 bytes) 2024-09-21T15:33:57.893 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:57.893+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:57.896 INFO:teuthology.orchestra.run.smithi107.stdout:3093 trunc from 0xc5afc9a to 0x78c6a6 2024-09-21T15:33:57.933 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:57.933 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:57.933 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:57.936 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:33:57.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:33:57.951 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.951 INFO:teuthology.orchestra.run.smithi107.stdout:3095 read 0x636a62 thru 0x6376cb (0xc6a bytes) 2024-09-21T15:33:57.963 INFO:teuthology.orchestra.run.smithi107.stdout:3096 trunc from 0x78c6a6 to 0x6e0df18 2024-09-21T15:33:57.970 INFO:teuthology.orchestra.run.smithi107.stdout:3097 write 0x43118ef thru 0x4315183 (0x3895 bytes) 2024-09-21T15:33:57.977 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:57.988 INFO:teuthology.orchestra.run.smithi107.stdout:3099 write 0x9c6c08f thru 0x9c75d0e (0x9c80 bytes) 2024-09-21T15:33:57.994 INFO:teuthology.orchestra.run.smithi107.stdout:3100 punch from 0x1cae43e to 0x1cb3fa4, (0x5b66 bytes) 2024-09-21T15:33:57.994 INFO:teuthology.orchestra.run.smithi107.stdout:3101 punch from 0x2eb1266 to 0x2ebe4dc, (0xd276 bytes) 2024-09-21T15:33:57.994 INFO:teuthology.orchestra.run.smithi107.stdout:3103 write 0x755c5b4 thru 0x75669c4 (0xa411 bytes) 2024-09-21T15:33:57.996 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:57.997+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:58.002 INFO:teuthology.orchestra.run.smithi107.stdout:3104 writesame 0x123a458 thru 0x1243a8f data_size 0x98(0x9638 bytes) 2024-09-21T15:33:58.009 INFO:teuthology.orchestra.run.smithi107.stdout:3106 read 0x721c42f thru 0x7220f10 (0x4ae2 bytes) 2024-09-21T15:33:58.009 INFO:teuthology.orchestra.run.smithi107.stdout:3107 read 0x379cd27 thru 0x37a9f20 (0xd1fa bytes) 2024-09-21T15:33:58.026 INFO:teuthology.orchestra.run.smithi107.stdout:3108 write 0xe498ff9 thru 0xe4a5675 (0xc67d bytes) 2024-09-21T15:33:58.034 INFO:teuthology.orchestra.run.smithi107.stdout:3109 write 0x941a91a thru 0x94275b1 (0xcc98 bytes) 2024-09-21T15:33:58.055 INFO:teuthology.orchestra.run.smithi107.stdout:3110 trunc from 0xe4a5676 to 0x86e6f6b 2024-09-21T15:33:58.077 INFO:teuthology.orchestra.run.smithi107.stdout:3111 write 0x44c5cd6 thru 0x44c7f3d (0x2268 bytes) 2024-09-21T15:33:58.080 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:58.090 INFO:teuthology.orchestra.run.smithi107.stdout:3114 write 0x9324aaa thru 0x9333788 (0xecdf bytes) 2024-09-21T15:33:58.094 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:58.093+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:58.112 INFO:teuthology.orchestra.run.smithi107.stdout:3115 trunc from 0x9333789 to 0x2a7e8ad 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:33:58.128 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.662851144000001, 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:33:58.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:33:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:58.132 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 47.432573767999997, 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:33:58.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:33:58.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:58.138 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 47.131202977000001, 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:33:58.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:33:58.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.143 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 46.455689157000002, 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:33:58.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:33:58.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126832794999999, 2024-09-21T15:33:58.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:33:58.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:33:58.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126816302999998, 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.153 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:33:58.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126799931000001, 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:33:58.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:33:58.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.161 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126783266, 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:33:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.165 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126766444000001, 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:33:58.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:33:58.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126438526000001, 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.170 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:58.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:33:58.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:33:58.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:33:58.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126407883999999, 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.175 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:33:58.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.179 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.12638974, 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.180 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:33:58.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126359283999999, 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.184 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:33:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:33:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126340996, 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:33:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:33:58.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.193 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126323708000001, 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126307042000001, 2024-09-21T15:33:58.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:33:58.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:33:58.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126282695, 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:33:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126266553000001, 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:33:58.207 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:33:58.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126250008, 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:33:58.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:58.216 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.126222464000001, 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:33:58.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:33:58.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.221 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:58.222 INFO:teuthology.orchestra.run.smithi107.stdout:3116 write 0x6d46192 thru 0x6d495c5 (0x3434 bytes) 2024-09-21T15:33:58.222 INFO:teuthology.orchestra.run.smithi107.stdout:3117 read 0x4b7f83a thru 0x4b88f92 (0x9759 bytes) 2024-09-21T15:33:58.223 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:33:58.223 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:58.219+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:58.324 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:58.319+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:58.391 INFO:teuthology.orchestra.run.smithi107.stdout:3118 writesame 0xdb7bb90 thru 0xdb8a643 data_size 0x12(0xeab4 bytes) 2024-09-21T15:33:58.391 INFO:teuthology.orchestra.run.smithi107.stdout:3120 read 0xd3dbbe7 thru 0xd3e7adb (0xbef5 bytes) 2024-09-21T15:33:58.391 INFO:teuthology.orchestra.run.smithi107.stdout:3121 punch from 0x8007eb1 to 0x800f74c, (0x789b bytes) 2024-09-21T15:33:58.391 INFO:teuthology.orchestra.run.smithi107.stdout:3123 write 0x8760f8 thru 0x880b0b (0xaa14 bytes) 2024-09-21T15:33:58.395 INFO:teuthology.orchestra.run.smithi107.stdout:3124 read 0x6666fd8 thru 0x6675fab (0xefd4 bytes) 2024-09-21T15:33:58.396 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:58.396 INFO:teuthology.orchestra.run.smithi107.stdout:3127 write 0x3358a2c thru 0x3361d52 (0x9327 bytes) 2024-09-21T15:33:58.397 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:58.398 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:33:58.398 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:58.407 INFO:teuthology.orchestra.run.smithi107.stdout:3128 writesame 0x6bccfe1 thru 0x6bd4063 data_size 0x1(0x7083 bytes) 2024-09-21T15:33:58.414 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:33:58.424 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:58.423+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:58.444 INFO:teuthology.orchestra.run.smithi107.stdout:3129 writesame 0x9422d80 thru 0x942e166 data_size 0x3d(0xb3e7 bytes) 2024-09-21T15:33:58.460 INFO:teuthology.orchestra.run.smithi107.stdout:3130 punch from 0x3b1ffd to 0x3b8e11, (0x6e14 bytes) 2024-09-21T15:33:58.460 INFO:teuthology.orchestra.run.smithi107.stdout:3133 write 0x1a0c3f0 thru 0x1a0f8eb (0x34fc bytes) 2024-09-21T15:33:58.465 INFO:teuthology.orchestra.run.smithi107.stdout:3134 write 0xe1bb32 thru 0xe1c739 (0xc08 bytes) 2024-09-21T15:33:58.468 INFO:teuthology.orchestra.run.smithi107.stdout:3135 punch from 0x43c0f17 to 0x43cbc36, (0xad1f bytes) 2024-09-21T15:33:58.468 INFO:teuthology.orchestra.run.smithi107.stdout:3136 writesame 0x979ba8f thru 0x97a2805 data_size 0x1(0x6d77 bytes) 2024-09-21T15:33:58.482 INFO:teuthology.orchestra.run.smithi107.stdout:3138 read 0x50df6d1 thru 0x50ea6cb (0xaffb bytes) 2024-09-21T15:33:58.496 INFO:teuthology.orchestra.run.smithi107.stdout:3139 writesame 0xe477e45 thru 0xe47f089 data_size 0x31(0x7245 bytes) 2024-09-21T15:33:58.496 INFO:teuthology.orchestra.run.smithi107.stdout:3140 read 0x39451d1 thru 0x394bfa1 (0x6dd1 bytes) 2024-09-21T15:33:58.496 INFO:teuthology.orchestra.run.smithi107.stdout:3141 read 0xd2e2dfe thru 0xd2e898d (0x5b90 bytes) 2024-09-21T15:33:58.496 INFO:teuthology.orchestra.run.smithi107.stdout:3142 write 0x89e7870 thru 0x89ec413 (0x4ba4 bytes) 2024-09-21T15:33:58.507 INFO:teuthology.orchestra.run.smithi107.stdout:3143 writesame 0x443294c thru 0x4439337 data_size 0x2(0x69ec bytes) 2024-09-21T15:33:58.525 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:58.525+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:58.526 INFO:teuthology.orchestra.run.smithi107.stdout:3144 read 0x4430699 thru 0x4430fbc (0x924 bytes) 2024-09-21T15:33:58.527 INFO:teuthology.orchestra.run.smithi107.stdout:3145 writesame 0xde4c898 thru 0xde51428 data_size 0x35(0x4b91 bytes) 2024-09-21T15:33:58.535 INFO:teuthology.orchestra.run.smithi107.stdout:3146 read 0x78c132c thru 0x78c5468 (0x413d bytes) 2024-09-21T15:33:58.535 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:58.535 INFO:teuthology.orchestra.run.smithi107.stdout:3149 read 0x90f0425 thru 0x90f639d (0x5f79 bytes) 2024-09-21T15:33:58.536 INFO:teuthology.orchestra.run.smithi107.stdout:3150 punch from 0x7630a94 to 0x7632c19, (0x2185 bytes) 2024-09-21T15:33:58.536 INFO:teuthology.orchestra.run.smithi107.stdout:3151 write 0x5727346 thru 0x573231e (0xafd9 bytes) 2024-09-21T15:33:58.538 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:58.538 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:33:58.538 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:33:58.538 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:33:58.538 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:58.554 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:33:58.573 INFO:teuthology.orchestra.run.smithi107.stdout:3152 trunc from 0xe47f08a to 0x4fd261a 2024-09-21T15:33:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:3153 punch from 0x47b2f93 to 0x47bba67, (0x8ad4 bytes) 2024-09-21T15:33:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:3155 read 0x4d71a66 thru 0x4d7a92d (0x8ec8 bytes) 2024-09-21T15:33:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:3156 read 0x20be5c0 thru 0x20ca06e (0xbaaf bytes) 2024-09-21T15:33:58.619 INFO:teuthology.orchestra.run.smithi107.stdout:3157 write 0x55a1bcf thru 0x55adb2e (0xbf60 bytes) 2024-09-21T15:33:58.641 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:58.641+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:58.647 INFO:teuthology.orchestra.run.smithi107.stdout:3158 writesame 0x8415db3 thru 0x8420f53 data_size 0x1(0xb1a1 bytes) 2024-09-21T15:33:58.647 INFO:teuthology.orchestra.run.smithi107.stdout:3160 read 0x4313931 thru 0x431d2a9 (0x9979 bytes) 2024-09-21T15:33:58.648 INFO:teuthology.orchestra.run.smithi107.stdout:3161 read 0x7dc409a thru 0x7dc5dd6 (0x1d3d bytes) 2024-09-21T15:33:58.649 INFO:teuthology.orchestra.run.smithi107.stdout:3162 write 0x24877f9 thru 0x2491a0c (0xa214 bytes) 2024-09-21T15:33:58.658 INFO:teuthology.orchestra.run.smithi107.stdout:3163 punch from 0xe7aa52 to 0xe80fcd, (0x657b bytes) 2024-09-21T15:33:58.659 INFO:teuthology.orchestra.run.smithi107.stdout:3164 read 0x4265a83 thru 0x4270c44 (0xb1c2 bytes) 2024-09-21T15:33:58.660 INFO:teuthology.orchestra.run.smithi107.stdout:3165 read 0x2a9630 thru 0x2b0bd5 (0x75a6 bytes) 2024-09-21T15:33:58.661 INFO:teuthology.orchestra.run.smithi107.stdout:3166 write 0x80d6ee4 thru 0x80e3f6f (0xd08c bytes) 2024-09-21T15:33:58.675 INFO:teuthology.orchestra.run.smithi107.stdout:3167 read 0x82e91ea thru 0x82eafd1 (0x1de8 bytes) 2024-09-21T15:33:58.676 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:58.694 INFO:teuthology.orchestra.run.smithi107.stdout:3169 write 0xa43bc5c thru 0xa43dd4f (0x20f4 bytes) 2024-09-21T15:33:58.702 INFO:teuthology.orchestra.run.smithi107.stdout:3171 write 0x36b3c65 thru 0x36bddc4 (0xa160 bytes) 2024-09-21T15:33:58.716 INFO:teuthology.orchestra.run.smithi107.stdout:3172 clone 24 order 20 su 16384 sc 11 2024-09-21T15:33:58.727 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:58.723+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:33:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9483875980000001, 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:33:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9480096069999999, 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:33:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:33:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:33:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9477903520000002, 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:33:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:33:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.947393189, 2024-09-21T15:33:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:33:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9469894009999997, 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:33:58.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9465336390000001, 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.750 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.939307093, 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:33:58.754 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:33:58.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9389408050000001, 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:33:58.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:33:58.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9382289620000002, 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.763 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:33:58.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:33:58.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9349684150000002, 2024-09-21T15:33:58.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:33:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.934528196, 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:33:58.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9342167730000002, 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.776 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:33:58.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:33:58.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9092891650000001, 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:33:58.780 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:33:58.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.908659509, 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:33:58.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9082550459999998, 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.789 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.907716653, 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.793 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9071896349999999, 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:33:58.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:33:58.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9067679100000001, 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.802 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:33:58.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:33:58.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.9063525079999999, 2024-09-21T15:33:58.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:33:58.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:33:58.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 3.905802081, 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:33:58.815 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:33:58.815 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:33:58.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:33:58.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:33:58.816 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:33:58.816 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:33:58.816 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-21T15:33:58.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T15:33:58.864 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:33:58.863+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:33:58.929 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:33:58.929+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:33:58.955 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:58.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:58.955 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:58.956 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:58.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:33:59.068 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:33:59.067+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:33:59.130 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:59.129+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:59.142 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:59.143 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:59.143 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:33:59.143 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:33:59.143 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:59.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:33:59.172 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T15:33:59.231 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:59.229+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:59.297 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:33:59.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 60.585992400000002, 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:33:59.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:33:59.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:33:59.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:33:59.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:59.303 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 60.575670105, 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:33:59.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 57.648172879999997, 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:33:59.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:33:59.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 57.614261446, 2024-09-21T15:33:59.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:33:59.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:33:59.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10948 3.1 3:82fda950:::rbd_data.11917b2c111e.0000000000000011:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:33:59.317 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.614298716, 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025101894999999999, 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10948 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:59.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681359+0000", 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681370+0000", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.135e-05 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681391+0000", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0633e-05 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681400+0000", 2024-09-21T15:33:59.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8109999999999992e-06 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681492+0000", 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2101000000000005e-05 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681709+0000", 2024-09-21T15:33:59.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021704800000000001 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.687063+0000", 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053539759999999999 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706359+0000", 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019296002 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706374+0000", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5245999999999999e-05 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706424+0000", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9996e-05 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706463+0000", 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9069999999999997e-05 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.323 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11973 3.18 3:1bdac4bd:::rbd_data.11917b2c111e.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 44.194331443999999, 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019588250000000001, 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11973 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.324 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101326+0000", 2024-09-21T15:33:59.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101338+0000", 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2381000000000001e-05 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101361+0000", 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2884e-05 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101372+0000", 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0716999999999999e-05 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101427+0000", 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4626000000000002e-05 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101493+0000", 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6115e-05 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.104640+0000", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003147501 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120784+0000", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016143372999999999 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120799+0000", 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5832999999999999e-05 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120871+0000", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1977000000000006e-05 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120917+0000", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5269999999999999e-05 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16569 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307060918000001, 2024-09-21T15:33:59.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016416218, 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16569 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988600+0000", 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3519999999999999e-06 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988603+0000", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4070000000000002e-06 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988606+0000", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7350000000000001e-06 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004678+0000", 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016071621000000001 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004772+0000", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4575999999999997e-05 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005015+0000", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000242752 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16570 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:59.333 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307051486999999, 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016891402999999999, 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16570 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988609+0000", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3659999999999999e-06 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988612+0000", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988615+0000", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5160000000000001e-06 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005071+0000", 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016456487999999998 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005205+0000", 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133638 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005500+0000", 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029470199999999998 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16571 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307042919000001, 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017340120000000001, 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16571 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988618+0000", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2559999999999999e-06 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988622+0000", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.087e-06 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988626+0000", 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1559999999999997e-06 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005524+0000", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016897760000000001 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005594+0000", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9998000000000003e-05 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005957+0000", 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036308 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.341 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16572 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307030025, 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017681184999999999, 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16572 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:33:59.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988632+0000", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.227e-06 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988638+0000", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1990000000000003e-06 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988641+0000", 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.835e-06 2024-09-21T15:33:59.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005986+0000", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017345098999999999 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006062+0000", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5685999999999993e-05 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006311+0000", 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000249452 2024-09-21T15:33:59.345 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16574 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307016796999999, 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956159999999999, 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16574 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988644+0000", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.35e-06 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988647+0000", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1750000000000001e-06 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988650+0000", 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.553e-06 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006336+0000", 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017685671 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006376+0000", 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0154999999999998e-05 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006599+0000", 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022348800000000001 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16575 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:59.350 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.307006515000001, 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01820097, 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16575 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988654+0000", 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1930000000000001e-06 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988657+0000", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.016e-06 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988661+0000", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2200000000000001e-06 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006625+0000", 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017964285 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006669+0000", 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4363000000000002e-05 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006854+0000", 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000185105 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16726 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306622034, 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016361140999999999, 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16726 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989038+0000", 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989039+0000", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.114e-06 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989062+0000", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2787000000000001e-05 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989066+0000", 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0999999999999997e-06 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005228+0000", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016161933999999999 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005298+0000", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0092999999999995e-05 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005399+0000", 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010131700000000001 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.358 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16729 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306591324999999, 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016513967000000001, 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16729 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.359 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989068+0000", 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989070+0000", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.809e-06 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989076+0000", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0530000000000003e-06 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989083+0000", 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9450000000000004e-06 2024-09-21T15:33:59.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005427+0000", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016344145000000001 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005495+0000", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8040999999999994e-05 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005583+0000", 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7250000000000007e-05 2024-09-21T15:33:59.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16732 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306573599, 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016636597999999999, 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16732 2024-09-21T15:33:59.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989091+0000", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4739999999999996e-06 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989097+0000", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9909999999999999e-06 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989102+0000", 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.792e-06 2024-09-21T15:33:59.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005607+0000", 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016505407 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005670+0000", 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2288999999999999e-05 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005723+0000", 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3154e-05 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16735 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306553911000002, 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016763145, 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16735 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989105+0000", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989108+0000", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7149999999999998e-06 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989123+0000", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5291999999999999e-05 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989128+0000", 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005749+0000", 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016620945000000002 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005806+0000", 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6436e-05 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005869+0000", 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3303e-05 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16738 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306527005, 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016893127000000001, 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16738 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989132+0000", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989135+0000", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4530000000000001e-06 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989141+0000", 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0599999999999996e-06 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989146+0000", 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005907+0000", 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016761426999999999 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005977+0000", 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0178000000000005e-05 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006026+0000", 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8862000000000001e-05 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16740 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 25.306510353, 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016975200999999999, 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16740 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989149+0000", 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989151+0000", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2170000000000001e-06 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989157+0000", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.631e-06 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989161+0000", 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4700000000000004e-06 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006040+0000", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016878955000000001 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006077+0000", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6838999999999997e-05 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006125+0000", 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7547999999999999e-05 2024-09-21T15:33:59.379 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:103668 3.b 3:d0264c1b:::rbd_data.1191b30e35a1.000000000000002e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88091227999999999, 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023214459999999999, 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 103668 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.380 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414746+0000", 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:33:59.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414758+0000", 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.255e-05 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414790+0000", 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1846e-05 2024-09-21T15:33:59.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414807+0000", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7054e-05 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414928+0000", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120555 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.415132+0000", 2024-09-21T15:33:59.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020484499999999999 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.419576+0000", 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044434260000000003 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437770+0000", 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018194248 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437781+0000", 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1147e-05 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437857+0000", 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5519000000000007e-05 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437962+0000", 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010523599999999999 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.385 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:103669 3.1b 3:dee5ef32:::rbd_data.1191b30e35a1.000000000000002f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88080784700000003, 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030556359000000002, 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 103669 2024-09-21T15:33:59.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:33:59.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414851+0000", 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414858+0000", 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3130000000000001e-06 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414872+0000", 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3529999999999999e-05 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414881+0000", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8899999999999996e-06 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414997+0000", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116709 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.415171+0000", 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017392800000000001 2024-09-21T15:33:59.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.421427+0000", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062557539999999997 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445200+0000", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023773424000000001 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445223+0000", 2024-09-21T15:33:59.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2442999999999998e-05 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445302+0000", 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9546e-05 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445408+0000", 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010597199999999999 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.391 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.392 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:33:59.392 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:33:59.392 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:59.392 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:33:59.394 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:33:59.394 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:59.391+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:33:59.471 INFO:teuthology.orchestra.run.smithi107.stdout:3176 writesame 0x8978f89 thru 0x8981a32 data_size 0x1(0x8aaa bytes) 2024-09-21T15:33:59.494 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:33:59.493+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:33:59.503 INFO:teuthology.orchestra.run.smithi107.stdout:3177 punch from 0x65ab1d6 to 0x65b71a5, (0xbfcf bytes) 2024-09-21T15:33:59.503 INFO:teuthology.orchestra.run.smithi107.stdout:3178 read 0x5d64686 thru 0x5d6d6d7 (0x9052 bytes) 2024-09-21T15:33:59.503 INFO:teuthology.orchestra.run.smithi107.stdout:3179 read 0x9061822 thru 0x9068f9f (0x777e bytes) 2024-09-21T15:33:59.504 INFO:teuthology.orchestra.run.smithi107.stdout:3180 write 0x37efa06 thru 0x37f8ac5 (0x90c0 bytes) 2024-09-21T15:33:59.513 INFO:teuthology.orchestra.run.smithi107.stdout:3181 writesame 0x4759912 thru 0x476373c data_size 0x21(0x9e2b bytes) 2024-09-21T15:33:59.528 INFO:teuthology.orchestra.run.smithi107.stdout:3183 trunc from 0xa43dd50 to 0x6224d8b 2024-09-21T15:33:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:33:59.579 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:33:59.579 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:33:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:33:59.584 INFO:teuthology.orchestra.run.smithi107.stdout:3184 write 0xbebfd6b thru 0xbec692a (0x6bc0 bytes) 2024-09-21T15:33:59.586 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:33:59.586 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:33:59.592 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.592 INFO:teuthology.orchestra.run.smithi107.stdout:3186 writesame 0x41009bf thru 0x4110862 data_size 0x4(0xfea4 bytes) 2024-09-21T15:33:59.596 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:33:59.593+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:33:59.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:33:59.605 INFO:teuthology.orchestra.run.smithi107.stdout:3187 trunc from 0xbec692b to 0xaed3185 2024-09-21T15:33:59.606 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:33:59.606 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:33:59.625 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:33:59.625 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:33:59.626 INFO:teuthology.orchestra.run.smithi107.stdout:3188 writesame 0x43f3ce2 thru 0x43fd4bb data_size 0x5d(0x97da bytes) 2024-09-21T15:33:59.635 INFO:teuthology.orchestra.run.smithi107.stdout:3189 read 0x609a152 thru 0x609f290 (0x513f bytes) 2024-09-21T15:33:59.636 INFO:teuthology.orchestra.run.smithi107.stdout:3193 read 0x118bed5 thru 0x119af42 (0xf06e bytes) 2024-09-21T15:33:59.644 INFO:teuthology.orchestra.run.smithi107.stdout:3195 trunc from 0xaed3185 to 0x6be2140 2024-09-21T15:33:59.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:33:59.657 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:33:59.676 INFO:teuthology.orchestra.run.smithi107.stdout:3196 write 0xe8c4679 thru 0xe8d39ca (0xf352 bytes) 2024-09-21T15:33:59.679 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:33:59.680 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:33:59.686 INFO:teuthology.orchestra.run.smithi107.stdout:3198 writesame 0x853b0e thru 0x860658 data_size 0x1(0xcb4b bytes) 2024-09-21T15:33:59.696 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:59.695+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:59.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:33:59.706 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:33:59.728 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:33:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:33:59.737 INFO:teuthology.orchestra.run.smithi107.stdout:3199 write 0x3fb7743 thru 0x3fbafb6 (0x3874 bytes) 2024-09-21T15:33:59.739 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:59.739 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:59.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:33:59.750 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:33:59.755 INFO:teuthology.orchestra.run.smithi107.stdout:3200 trunc from 0xe8d39cb to 0x7afd521 2024-09-21T15:33:59.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:33:59.774 INFO:teuthology.orchestra.run.smithi107.stdout:3201 write 0x44f63ff thru 0x44fa061 (0x3c63 bytes) 2024-09-21T15:33:59.778 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.778 INFO:teuthology.orchestra.run.smithi107.stdout:3204 writesame 0x729c1e8 thru 0x72a8c95 data_size 0x1(0xcaae bytes) 2024-09-21T15:33:59.797 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:33:59.795+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:33:59.813 INFO:teuthology.orchestra.run.smithi107.stdout:3205 read 0x2379bc2 thru 0x2388dbd (0xf1fc bytes) 2024-09-21T15:33:59.814 INFO:teuthology.orchestra.run.smithi107.stdout:3206 read 0x3b130b0 thru 0x3b1dfbf (0xaf10 bytes) 2024-09-21T15:33:59.816 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.816 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.816 INFO:teuthology.orchestra.run.smithi107.stdout:3209 punch from 0x1975e15 to 0x1980452, (0xa63d bytes) 2024-09-21T15:33:59.826 INFO:teuthology.orchestra.run.smithi107.stdout:3210 write 0x7becb1a thru 0x7bf4a73 (0x7f5a bytes) 2024-09-21T15:33:59.834 INFO:teuthology.orchestra.run.smithi107.stdout:3211 read 0x799f8f6 thru 0x79ac112 (0xc81d bytes) 2024-09-21T15:33:59.840 INFO:teuthology.orchestra.run.smithi107.stdout:3212 trunc from 0x7bf4a74 to 0xc12c49e 2024-09-21T15:33:59.851 INFO:teuthology.orchestra.run.smithi107.stdout:3213 read 0x62b8f9 thru 0x63a119 (0xe821 bytes) 2024-09-21T15:33:59.853 INFO:teuthology.orchestra.run.smithi107.stdout:3215 read 0x9db8f01 thru 0x9dc075c (0x785c bytes) 2024-09-21T15:33:59.853 INFO:teuthology.orchestra.run.smithi107.stdout:3216 write 0x8bd55d6 thru 0x8bdb949 (0x6374 bytes) 2024-09-21T15:33:59.861 INFO:teuthology.orchestra.run.smithi107.stdout:3217 punch from 0x30fadd6 to 0x30fed6c, (0x3f96 bytes) 2024-09-21T15:33:59.862 INFO:teuthology.orchestra.run.smithi107.stdout:3218 read 0x4dcea65 thru 0x4dd39f2 (0x4f8e bytes) 2024-09-21T15:33:59.862 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.862 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.862 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.863 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.863 INFO:teuthology.orchestra.run.smithi107.stdout:3225 read 0x1516cc1 thru 0x151b249 (0x4589 bytes) 2024-09-21T15:33:59.876 INFO:teuthology.orchestra.run.smithi107.stdout:3228 write 0xcd0ad58 thru 0xcd0f121 (0x43ca bytes) 2024-09-21T15:33:59.885 INFO:teuthology.orchestra.run.smithi107.stdout:3230 read 0xa67f29d thru 0xa682e02 (0x3b66 bytes) 2024-09-21T15:33:59.885 INFO:teuthology.orchestra.run.smithi107.stdout:3232 punch from 0x4912360 to 0x491d2ec, (0xaf8c bytes) 2024-09-21T15:33:59.885 INFO:teuthology.orchestra.run.smithi107.stdout:3233 write 0x20ee783 thru 0x20ef6d8 (0xf56 bytes) 2024-09-21T15:33:59.892 INFO:teuthology.orchestra.run.smithi107.stdout:3234 write 0x58a7c35 thru 0x58b29d1 (0xad9d bytes) 2024-09-21T15:33:59.895 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:33:59.895 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:33:59.898 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:33:59.897+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:33:59.912 INFO:teuthology.orchestra.run.smithi107.stdout:3235 trunc from 0xcd0f122 to 0x58102c9 2024-09-21T15:33:59.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:33:59.954 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T15:33:59.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:33:59.977 INFO:teuthology.orchestra.run.smithi107.stdout:3236 writesame 0x71f7db4 thru 0x72032b3 data_size 0x40(0xb500 bytes) 2024-09-21T15:33:59.977 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:33:59.989 INFO:teuthology.orchestra.run.smithi107.stdout:3238 write 0x7f2a99b thru 0x7f34c4a (0xa2b0 bytes) 2024-09-21T15:33:59.998 INFO:teuthology.orchestra.run.smithi107.stdout:3239 write 0x1aac9df thru 0x1ab28e1 (0x5f03 bytes) 2024-09-21T15:33:59.999 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:33:59.995+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:00.010 INFO:teuthology.orchestra.run.smithi107.stdout:3240 punch from 0x7684970 to 0x768de13, (0x94a3 bytes) 2024-09-21T15:34:00.018 INFO:teuthology.orchestra.run.smithi107.stdout:3241 trunc from 0x7f34c4b to 0xd428fab 2024-09-21T15:34:00.029 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.029 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.029 INFO:teuthology.orchestra.run.smithi107.stdout:3244 punch from 0xc7ddeb4 to 0xc7e5f1a, (0x8066 bytes) 2024-09-21T15:34:00.029 INFO:teuthology.orchestra.run.smithi107.stdout:3245 read 0x62c719 thru 0x636d50 (0xa638 bytes) 2024-09-21T15:34:00.030 INFO:teuthology.orchestra.run.smithi107.stdout:3247 read 0x1bf78f thru 0x1c6404 (0x6c76 bytes) 2024-09-21T15:34:00.036 INFO:teuthology.orchestra.run.smithi107.stdout:3250 write 0xc427122 thru 0xc42910b (0x1fea bytes) 2024-09-21T15:34:00.044 INFO:teuthology.orchestra.run.smithi107.stdout:3251 punch from 0xa1b615b to 0xa1b6b57, (0x9fc bytes) 2024-09-21T15:34:00.044 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.057 INFO:teuthology.orchestra.run.smithi107.stdout:3254 trunc from 0xd428fab to 0x457af45 2024-09-21T15:34:00.084 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.084 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.092 INFO:teuthology.orchestra.run.smithi107.stdout:3255 write 0x202a17a thru 0x202eb93 (0x4a1a bytes) 2024-09-21T15:34:00.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:00.100 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:00.095+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:00.103 INFO:teuthology.orchestra.run.smithi107.stdout:3257 trunc from 0x457af45 to 0x7a5e412 2024-09-21T15:34:00.112 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.112 INFO:teuthology.orchestra.run.smithi107.stdout:3261 read 0x173e9f4 thru 0x174b67e (0xcc8b bytes) 2024-09-21T15:34:00.133 INFO:teuthology.orchestra.run.smithi107.stdout:3262 write 0xea06a96 thru 0xea0ef82 (0x84ed bytes) 2024-09-21T15:34:00.140 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.140 INFO:teuthology.orchestra.run.smithi107.stdout:3265 read 0xc4bfde2 thru 0xc4ca371 (0xa590 bytes) 2024-09-21T15:34:00.140 INFO:teuthology.orchestra.run.smithi107.stdout:3266 read 0xb6e6f65 thru 0xb6f39cd (0xca69 bytes) 2024-09-21T15:34:00.140 INFO:teuthology.orchestra.run.smithi107.stdout:3269 write 0x7acc0d7 thru 0x7ad4ccc (0x8bf6 bytes) 2024-09-21T15:34:00.147 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.148 INFO:teuthology.orchestra.run.smithi107.stdout:3271 write 0x5444f55 thru 0x5452407 (0xd4b3 bytes) 2024-09-21T15:34:00.156 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.156 INFO:teuthology.orchestra.run.smithi107.stdout:3274 write 0xe7a0e21 thru 0xe7a1c97 (0xe77 bytes) 2024-09-21T15:34:00.158 INFO:teuthology.orchestra.run.smithi107.stdout:3277 write 0x2a59a79 thru 0x2a65b11 (0xc099 bytes) 2024-09-21T15:34:00.167 INFO:teuthology.orchestra.run.smithi107.stdout:3278 read 0x57f2c9e thru 0x57fbc5a (0x8fbd bytes) 2024-09-21T15:34:00.167 INFO:teuthology.orchestra.run.smithi107.stdout:3281 punch from 0x1fb916f to 0x1fbd922, (0x47b3 bytes) 2024-09-21T15:34:00.168 INFO:teuthology.orchestra.run.smithi107.stdout:3282 writesame 0x97ca78c thru 0x97cab3e data_size 0x1(0x3b3 bytes) 2024-09-21T15:34:00.173 INFO:teuthology.orchestra.run.smithi107.stdout:3283 read 0xe5e945 thru 0xe5ebe9 (0x2a5 bytes) 2024-09-21T15:34:00.174 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.174 INFO:teuthology.orchestra.run.smithi107.stdout:3287 writesame 0x9d5667b thru 0x9d5e5f3 data_size 0x1(0x7f79 bytes) 2024-09-21T15:34:00.198 INFO:teuthology.orchestra.run.smithi107.stdout:3288 writesame 0x4d3ea8b thru 0x4d4d3ca data_size 0x18(0xe940 bytes) 2024-09-21T15:34:00.200 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:00.201+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:00.207 INFO:teuthology.orchestra.run.smithi107.stdout:3289 writesame 0x544946e thru 0x544b93e data_size 0x1d(0x24d1 bytes) 2024-09-21T15:34:00.210 INFO:teuthology.orchestra.run.smithi107.stdout:3290 write 0x2549a67 thru 0x255277b (0x8d15 bytes) 2024-09-21T15:34:00.229 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.229 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.229 INFO:teuthology.orchestra.run.smithi107.stdout:3291 trunc from 0xea0ef83 to 0x7e6807d 2024-09-21T15:34:00.244 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:3293 writesame 0x3d892c8 thru 0x3d94a6d data_size 0xb(0xb7a6 bytes) 2024-09-21T15:34:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:3295 writesame 0x33f9113 thru 0x3401b3a data_size 0x4(0x8a28 bytes) 2024-09-21T15:34:00.278 INFO:teuthology.orchestra.run.smithi107.stdout:3297 trunc from 0x7e6807d to 0x1335fed 2024-09-21T15:34:00.302 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:00.301+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:00.347 INFO:teuthology.orchestra.run.smithi107.stdout:3298 write 0x889529c thru 0x8899765 (0x44ca bytes) 2024-09-21T15:34:00.353 INFO:teuthology.orchestra.run.smithi107.stdout:3299 write 0x16c905d thru 0x16ca643 (0x15e7 bytes) 2024-09-21T15:34:00.357 INFO:teuthology.orchestra.run.smithi107.stdout:3300 writesame 0x823a386 thru 0x8249b8b data_size 0x35(0xf806 bytes) 2024-09-21T15:34:00.364 INFO:teuthology.orchestra.run.smithi107.stdout:3301 read 0x2cd2c1f thru 0x2cddd84 (0xb166 bytes) 2024-09-21T15:34:00.364 INFO:teuthology.orchestra.run.smithi107.stdout:3302 write 0x47bf8d2 thru 0x47c84b5 (0x8be4 bytes) 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:3305 punch from 0x369a3c5 to 0x369aa20, (0x65b bytes) 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:3306 punch from 0x475ef20 to 0x475efac, (0x8c bytes) 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:3309 punch from 0x8057e44 to 0x8061371, (0x952d bytes) 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:3310 punch from 0x20629ea to 0x2071b29, (0xf13f bytes) 2024-09-21T15:34:00.372 INFO:teuthology.orchestra.run.smithi107.stdout:3311 read 0x112256d thru 0x1126067 (0x3afb bytes) 2024-09-21T15:34:00.373 INFO:teuthology.orchestra.run.smithi107.stdout:3313 clone 25 order 20 su 32768 sc 8 2024-09-21T15:34:00.387 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.387 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.400 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:00.444 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:00.443+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:00.503 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:00.501+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:00.549 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.549 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:00.604 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:00.605+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.691 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:00.705 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:00.705+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:00.805 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:00.805+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:00.815 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.815 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:00.906 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:00.905+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:00.958 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:00.958 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:00.969 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:34:00.976 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:01.007 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:01.003+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:01.073 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone24 from 0x8899766 (overlap 0x1335fed) to 0x1ebcca 2024-09-21T15:34:01.107 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:01.103+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:01.162 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.162 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.177 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:01.208 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:01.207+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:01.213 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T15:34:01.298 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.298 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.310 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:01.309+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:01.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:01.410 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:01.407+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:01.481 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.481 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:01.541 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:01.541+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:01.612 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:01.611+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:01.612 INFO:teuthology.orchestra.run.smithi107.stdout:3314 writesame 0xc8b9ae3 thru 0xc8bcf5e data_size 0x2(0x347c bytes) 2024-09-21T15:34:01.612 INFO:teuthology.orchestra.run.smithi107.stdout:3315 write 0x17bffdf thru 0x17c494d (0x496f bytes) 2024-09-21T15:34:01.625 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.625 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.631 INFO:teuthology.orchestra.run.smithi107.stdout:3317 write 0xa5a8ad7 thru 0xa5b4f42 (0xc46c bytes) 2024-09-21T15:34:01.642 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:01.649 INFO:teuthology.orchestra.run.smithi107.stdout:3318 trunc from 0xc8bcf5f to 0x58f91b8 2024-09-21T15:34:01.680 INFO:teuthology.orchestra.run.smithi107.stdout:3319 write 0x3cdcb71 thru 0x3ce1751 (0x4be1 bytes) 2024-09-21T15:34:01.694 INFO:teuthology.orchestra.run.smithi107.stdout:3321 trunc from 0x58f91b8 to 0x2090e83 2024-09-21T15:34:01.713 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:01.711+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:01.720 INFO:teuthology.orchestra.run.smithi107.stdout:3322 write 0xf20973 thru 0xf2820b (0x7899 bytes) 2024-09-21T15:34:01.746 INFO:teuthology.orchestra.run.smithi107.stdout:3323 write 0x38ee193 thru 0x38ef80d (0x167b bytes) 2024-09-21T15:34:01.758 INFO:teuthology.orchestra.run.smithi107.stdout:3325 writesame 0x3ada738 thru 0x3adfc7c data_size 0x1(0x5545 bytes) 2024-09-21T15:34:01.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:01.775 INFO:teuthology.orchestra.run.smithi107.stdout:3330 write 0x7b811c8 thru 0x7b8c447 (0xb280 bytes) 2024-09-21T15:34:01.781 INFO:teuthology.orchestra.run.smithi107.stdout:3331 write 0x1b080b8 thru 0x1b0818f (0xd8 bytes) 2024-09-21T15:34:01.788 INFO:teuthology.orchestra.run.smithi107.stdout:3332 write 0x1589f88 thru 0x158e888 (0x4901 bytes) 2024-09-21T15:34:01.799 INFO:teuthology.orchestra.run.smithi107.stdout:3334 writesame 0x4a92c8c thru 0x4a959eb data_size 0x42(0x2d60 bytes) 2024-09-21T15:34:01.814 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:01.811+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:01.821 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.821 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.828 INFO:teuthology.orchestra.run.smithi107.stdout:3337 write 0xe3bc940 thru 0xe3bd663 (0xd24 bytes) 2024-09-21T15:34:01.833 INFO:teuthology.orchestra.run.smithi107.stdout:3338 read 0x8c48541 thru 0x8c5384f (0xb30f bytes) 2024-09-21T15:34:01.833 INFO:teuthology.orchestra.run.smithi107.stdout:3339 write 0x80f06c4 thru 0x80f682b (0x6168 bytes) 2024-09-21T15:34:01.837 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:01.839 INFO:teuthology.orchestra.run.smithi107.stdout:3341 read 0xb196c78 thru 0xb1a0685 (0x9a0e bytes) 2024-09-21T15:34:01.839 INFO:teuthology.orchestra.run.smithi107.stdout:3342 write 0x14657b0 thru 0x1474637 (0xee88 bytes) 2024-09-21T15:34:01.854 INFO:teuthology.orchestra.run.smithi107.stdout:3343 punch from 0x17f2fad to 0x17fcd05, (0x9d58 bytes) 2024-09-21T15:34:01.854 INFO:teuthology.orchestra.run.smithi107.stdout:3344 write 0x53a36b7 thru 0x53ab8fa (0x8244 bytes) 2024-09-21T15:34:01.859 INFO:teuthology.orchestra.run.smithi107.stdout:3345 read 0x8561bb4 thru 0x856b856 (0x9ca3 bytes) 2024-09-21T15:34:01.859 INFO:teuthology.orchestra.run.smithi107.stdout:3347 writesame 0x5bfb508 thru 0x5c02e5a data_size 0x15(0x7953 bytes) 2024-09-21T15:34:01.865 INFO:teuthology.orchestra.run.smithi107.stdout:3348 writesame 0x35e5de5 thru 0x35e5ee8 data_size 0xa(0x104 bytes) 2024-09-21T15:34:01.870 INFO:teuthology.orchestra.run.smithi107.stdout:3349 flatten 2024-09-21T15:34:01.915 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:01.912+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:01.948 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:01.955 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:01.960 INFO:teuthology.orchestra.run.smithi107.stdout:3352 trunc from 0xe3bd664 to 0xb4dc52a 2024-09-21T15:34:01.965 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:01.966 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:01.980 INFO:teuthology.orchestra.run.smithi107.stdout:3355 write 0xe502a5d thru 0xe5087c4 (0x5d68 bytes) 2024-09-21T15:34:01.985 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:01.985 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:01.995 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:01.995 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:01.996 INFO:teuthology.orchestra.run.smithi107.stdout:3357 trunc from 0xe5087c5 to 0x7a538fc 2024-09-21T15:34:02.007 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:02.008 INFO:teuthology.orchestra.run.smithi107.stdout:3361 writesame 0x471dbbb thru 0x472c5de data_size 0x14(0xea24 bytes) 2024-09-21T15:34:02.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:3362 writesame 0x6f18ba5 thru 0x6f1df96 data_size 0xe(0x53f2 bytes) 2024-09-21T15:34:02.015 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:02.011+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:02.016 INFO:teuthology.orchestra.run.smithi107.stdout:3363 read 0x3427643 thru 0x3431e8a (0xa848 bytes) 2024-09-21T15:34:02.017 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.017 INFO:teuthology.orchestra.run.smithi107.stdout:3368 write 0x2f5cc6f thru 0x2f6c502 (0xf894 bytes) 2024-09-21T15:34:02.022 INFO:teuthology.orchestra.run.smithi107.stdout:3369 read 0x67d8bcf thru 0x67e7fce (0xf400 bytes) 2024-09-21T15:34:02.023 INFO:teuthology.orchestra.run.smithi107.stdout:3370 punch from 0x32c412c to 0x32d34b5, (0xf389 bytes) 2024-09-21T15:34:02.023 INFO:teuthology.orchestra.run.smithi107.stdout:3374 read 0x4d9132d thru 0x4d91afa (0x7ce bytes) 2024-09-21T15:34:02.025 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:02.026 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:02.030 INFO:teuthology.orchestra.run.smithi107.stdout:3375 write 0x9a2a1c9 thru 0x9a37341 (0xd179 bytes) 2024-09-21T15:34:02.035 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.035 INFO:teuthology.orchestra.run.smithi107.stdout:3377 read 0x1990504 thru 0x19991cb (0x8cc8 bytes) 2024-09-21T15:34:02.035 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.040 INFO:teuthology.orchestra.run.smithi107.stdout:3379 trunc from 0x9a37342 to 0xce05811 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stdout:3380 read 0x1119a18 thru 0x111e306 (0x48ef bytes) 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stdout:3381 read 0xbe476dc thru 0xbe4d0f7 (0x5a1c bytes) 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stdout:3389 read 0x26902f2 thru 0x2697b1c (0x782b bytes) 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:02.046 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:02.047 INFO:teuthology.orchestra.run.smithi107.stdout:3390 write 0x25d02cf thru 0x25d7bd4 (0x7906 bytes) 2024-09-21T15:34:02.048 INFO:teuthology.orchestra.run.smithi107.stdout:3391 writesame 0x572d8a1 thru 0x5735b21 data_size 0x1(0x8281 bytes) 2024-09-21T15:34:02.063 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:02.063 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:02.063 INFO:teuthology.orchestra.run.smithi107.stdout:3392 write 0xa3f43b9 thru 0xa3fe5f4 (0xa23c bytes) 2024-09-21T15:34:02.075 INFO:teuthology.orchestra.run.smithi107.stdout:3395 trunc from 0xce05811 to 0x79ef6c0 2024-09-21T15:34:02.081 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:02.082 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:02.087 INFO:teuthology.orchestra.run.smithi107.stdout:3396 read 0x51531a6 thru 0x5159ed6 (0x6d31 bytes) 2024-09-21T15:34:02.087 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.087 INFO:teuthology.orchestra.run.smithi107.stdout:3399 read 0x394427e thru 0x3948375 (0x40f8 bytes) 2024-09-21T15:34:02.087 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.105 INFO:teuthology.orchestra.run.smithi107.stdout:3402 writesame 0xeaa3008 thru 0xeaa592f data_size 0xc(0x2928 bytes) 2024-09-21T15:34:02.106 INFO:teuthology.orchestra.run.smithi107.stdout:3403 punch from 0x3f7517e to 0x3f751fd, (0x7f bytes) 2024-09-21T15:34:02.122 INFO:teuthology.orchestra.run.smithi107.stdout:3405 trunc from 0xeaa5930 to 0x3dee6ae 2024-09-21T15:34:02.122 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:02.119+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:02.140 INFO:teuthology.orchestra.run.smithi107.stdout:3406 trunc from 0x3dee6ae to 0x5ee649b 2024-09-21T15:34:02.154 INFO:teuthology.orchestra.run.smithi107.stdout:3407 trunc from 0x5ee649b to 0x41eefd 2024-09-21T15:34:02.180 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:02.180 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:02.180 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:02.180 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:02.180 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:02.181 INFO:teuthology.orchestra.run.smithi107.stdout:3408 punch from 0x35feff to 0x36ca4d, (0xcb4e bytes) 2024-09-21T15:34:02.195 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:02.195 INFO:teuthology.orchestra.run.smithi107.stdout:3409 trunc from 0x41eefd to 0x8091dd3 2024-09-21T15:34:02.203 INFO:teuthology.orchestra.run.smithi107.stdout:3411 read 0x721adb7 thru 0x721d85a (0x2aa4 bytes) 2024-09-21T15:34:02.203 INFO:teuthology.orchestra.run.smithi107.stdout:3413 write 0x54f71d9 thru 0x5502503 (0xb32b bytes) 2024-09-21T15:34:02.219 INFO:teuthology.orchestra.run.smithi107.stdout:3414 write 0x89b759b thru 0x89c4960 (0xd3c6 bytes) 2024-09-21T15:34:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:3416 read 0x711664b thru 0x711c67a (0x6030 bytes) 2024-09-21T15:34:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.239 INFO:teuthology.orchestra.run.smithi107.stdout:3419 writesame 0xa806e1d thru 0xa80f1ae data_size 0xb(0x8392 bytes) 2024-09-21T15:34:02.240 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:02.240+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:02.256 INFO:teuthology.orchestra.run.smithi107.stdout:3420 trunc from 0xa80f1af to 0x37d177e 2024-09-21T15:34:02.289 INFO:teuthology.orchestra.run.smithi107.stdout:3421 writesame 0x7e562d7 thru 0x7e57689 data_size 0x3(0x13b3 bytes) 2024-09-21T15:34:02.313 INFO:teuthology.orchestra.run.smithi107.stdout:3422 writesame 0xc23d1ff thru 0xc241fc7 data_size 0x1(0x4dc9 bytes) 2024-09-21T15:34:02.313 INFO:teuthology.orchestra.run.smithi107.stdout:3423 write 0x33d1f19 thru 0x33d7936 (0x5a1e bytes) 2024-09-21T15:34:02.317 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:02.315+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:02.320 INFO:teuthology.orchestra.run.smithi107.stdout:3424 read 0x63aadf5 thru 0x63b9fb0 (0xf1bc bytes) 2024-09-21T15:34:02.320 INFO:teuthology.orchestra.run.smithi107.stdout:3425 trunc from 0xc241fc8 to 0xc139ec4 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_repop(client.4497.0:3825 3.1e e31/21 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head v 31'13, mlcod=28'11)", 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.861483555, 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042921882000000001, 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.329 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463315+0000", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.463312+0000", 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478950+0000", 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.015637961999999998 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.478999+0000", 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8384000000000003e-05 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479018+0000", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9106e-05 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479171+0000", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015273500000000001 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.479215+0000", 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.4777000000000002e-05 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506159+0000", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.026943165000000002 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:32:58.506237+0000", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7953000000000004e-05 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:9956 3.a 3:55bdc5ce:::rbd_data.1191f1341b63.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:34:02.333 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.631206179000003, 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.070171234999999998, 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 9956 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:34:02.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693592+0000", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693589+0000", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693604+0000", 2024-09-21T15:34:02.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4066000000000001e-05 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693637+0000", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3393999999999999e-05 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693655+0000", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7790000000000001e-05 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693750+0000", 2024-09-21T15:34:02.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5031999999999994e-05 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.693969+0000", 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000219022 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.706965+0000", 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.012996409 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763404+0000", 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.056439233999999998 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763566+0000", 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000161299 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763658+0000", 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2447000000000004e-05 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.763763+0000", 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010508199999999999 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10056 3.a 3:57424727:::rbd_data.1191f1341b63.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.329835387999999, 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.054572632000000003, 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:02.339 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10056 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994963+0000", 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994961+0000", 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.994974+0000", 2024-09-21T15:34:02.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.293e-05 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995005+0000", 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.146e-05 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995019+0000", 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.435e-05 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995156+0000", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000136382 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:10.995539+0000", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000383087 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.003417+0000", 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0078776390000000005 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049253+0000", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045836730999999999 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049365+0000", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000111665 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049467+0000", 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000101922 2024-09-21T15:34:02.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.049535+0000", 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8563000000000001e-05 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:10329 3.1e 3:7aa26167:::rbd_data.1191f1341b63.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 50.654321568, 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.057260433999999999, 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.345 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 10329 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670477+0000", 2024-09-21T15:34:02.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670474+0000", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670490+0000", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5869999999999999e-05 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670524+0000", 2024-09-21T15:34:02.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.4403000000000002e-05 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670539+0000", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4953000000000001e-05 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670663+0000", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012401299999999999 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.670928+0000", 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026462099999999999 2024-09-21T15:34:02.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 3", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.680731+0000", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0098035770000000008 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727561+0000", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.04682936 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727619+0000", 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.838e-05 2024-09-21T15:34:02.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727688+0000", 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9127000000000004e-05 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:11.727737+0000", 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9004e-05 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16772 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 270336~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325465206000001, 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042430514000000003, 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16772 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999333+0000", 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999335+0000", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0209999999999998e-06 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999341+0000", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.896e-06 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999345+0000", 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7859999999999996e-06 2024-09-21T15:34:02.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041617+0000", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042271290000000003 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041701+0000", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4809999999999996e-05 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041764+0000", 2024-09-21T15:34:02.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2046999999999997e-05 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16774 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 286720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325448714, 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.045703061000000003, 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.355 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16774 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999349+0000", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999351+0000", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1229999999999998e-06 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999357+0000", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.054e-06 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999362+0000", 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5220000000000001e-06 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041790+0000", 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042428186 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.041923+0000", 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013302600000000001 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.045053+0000", 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0031294959999999998 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16776 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 303104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325432341999999, 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042823859999999998, 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16776 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999366+0000", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999368+0000", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0099999999999998e-06 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999374+0000", 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0789999999999997e-06 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999378+0000", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7169999999999999e-06 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042031+0000", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042652164999999999 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042111+0000", 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0335999999999998e-05 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042190+0000", 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8841e-05 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16777 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 319488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325415676999999, 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042946312, 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16777 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999383+0000", 2024-09-21T15:34:02.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999382+0000", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999384+0000", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2879999999999999e-06 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999390+0000", 2024-09-21T15:34:02.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8680000000000004e-06 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999395+0000", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6770000000000003e-06 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042213+0000", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042817901999999998 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042284+0000", 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0623000000000005e-05 2024-09-21T15:34:02.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042329+0000", 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.5303999999999999e-05 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16778 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 335872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325398855, 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043089568000000002, 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.367 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16778 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:34:02.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999399+0000", 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999407+0000", 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0779999999999996e-06 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999423+0000", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6334999999999999e-05 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999718+0000", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029428199999999999 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042351+0000", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042633622000000003 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042427+0000", 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5642999999999994e-05 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042489+0000", 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1950000000000001e-05 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16779 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 352256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325070937, 2024-09-21T15:34:02.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042986812999999999, 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16779 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999727+0000", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999726+0000", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999736+0000", 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.8949999999999993e-06 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999746+0000", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3479999999999993e-06 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999753+0000", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9120000000000001e-06 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042510+0000", 2024-09-21T15:34:02.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042757876 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042664+0000", 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015354600000000001 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042714+0000", 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0065000000000003e-05 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16781 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 368640~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325040295000001, 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043042785, 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16781 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999758+0000", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999757+0000", 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999760+0000", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6740000000000001e-06 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999767+0000", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3790000000000003e-06 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999772+0000", 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9980000000000003e-06 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042728+0000", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.042956165999999997 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042769+0000", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0861000000000001e-05 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042801+0000", 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2159999999999997e-05 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16783 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 385024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.325022150999999, 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043192461000000001, 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16783 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.380 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999776+0000", 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999778+0000", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1890000000000001e-06 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999786+0000", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999799+0000", 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3105000000000001e-05 2024-09-21T15:34:02.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042820+0000", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043020267000000001 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042894+0000", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3874000000000005e-05 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042969+0000", 2024-09-21T15:34:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4894999999999994e-05 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16784 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 401408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.384 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:34:02.385 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324991695000001, 2024-09-21T15:34:02.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043367595000000002, 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16784 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999807+0000", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999806+0000", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999809+0000", 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5689999999999999e-06 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999815+0000", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4520000000000002e-06 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999820+0000", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8210000000000001e-06 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.042990+0000", 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043170274000000002 2024-09-21T15:34:02.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043069+0000", 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8481999999999998e-05 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043174+0000", 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000105406 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16786 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 417792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324973407000002, 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043512202999999999, 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16786 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999825+0000", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999824+0000", 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999827+0000", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2050000000000001e-06 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999833+0000", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9710000000000001e-06 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999838+0000", 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.9710000000000003e-06 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043197+0000", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043359705999999998 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043273+0000", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5463000000000006e-05 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043337+0000", 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4249000000000001e-05 2024-09-21T15:34:02.393 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16787 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 434176~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324956118999999, 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043654586000000002, 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.394 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16787 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999842+0000", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999844+0000", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1799999999999999e-06 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999850+0000", 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7629999999999997e-06 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999854+0000", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.3959999999999999e-06 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043359+0000", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043505414999999999 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043435+0000", 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5335999999999998e-05 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043497+0000", 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1860999999999996e-05 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16789 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 450560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324939452999999, 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043791010999999998, 2024-09-21T15:34:02.398 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16789 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999859+0000", 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999858+0000", 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999867+0000", 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4770000000000003e-06 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999874+0000", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7109999999999998e-06 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999878+0000", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043517+0000", 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043638986999999997 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043581+0000", 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3102000000000007e-05 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043650+0000", 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9227000000000007e-05 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.402 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16791 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324915105999999, 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044522094999999998, 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16791 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:34:02.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999883+0000", 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999885+0000", 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1320000000000001e-06 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999891+0000", 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7930000000000003e-06 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999895+0000", 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.6319999999999999e-06 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.043676+0000", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.043780762000000001 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044343+0000", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00066671299999999998 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044405+0000", 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2383000000000002e-05 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16792 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324898963999999, 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044662316000000001, 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16792 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999899+0000", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999901+0000", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.159e-06 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999907+0000", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7389999999999996e-06 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999912+0000", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.7469999999999996e-06 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044427+0000", 2024-09-21T15:34:02.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044515159999999998 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044500+0000", 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3266e-05 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044562+0000", 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.1591e-05 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16794 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324882419000001, 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044804683999999997, 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16794 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999916+0000", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999915+0000", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999918+0000", 2024-09-21T15:34:02.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2859999999999998e-06 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999924+0000", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8289999999999996e-06 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999938+0000", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4548e-05 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044586+0000", 2024-09-21T15:34:02.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044647421999999999 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044657+0000", 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1656999999999995e-05 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044720+0000", 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3291000000000001e-05 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:16795 3.1e 3:7b1d3cee:::rbd_data.1191dfb05674.0000000000000006:16 [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.324854875, 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044982266999999999, 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 16795 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999943+0000", 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999950+0000", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3300000000000001e-06 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999958+0000", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2570000000000001e-06 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:33.999963+0000", 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0830000000000003e-06 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044741+0000", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.044778608999999997 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044846+0000", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010483 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:34.044926+0000", 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9541000000000003e-05 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.418 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.419 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.419 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.419 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:02.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:02.421 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:02.416+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:3427 write 0x5189d31 thru 0x51977f3 (0xdac3 bytes) 2024-09-21T15:34:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:3428 punch from 0x8c73493 to 0x8c79806, (0x6373 bytes) 2024-09-21T15:34:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:3429 read 0x8fa921 thru 0x904903 (0x9fe3 bytes) 2024-09-21T15:34:02.422 INFO:teuthology.orchestra.run.smithi107.stdout:3430 read 0x3039b06 thru 0x303c3fc (0x28f7 bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:3432 writesame 0x6a1773a thru 0x6a1f2f2 data_size 0x1(0x7bb9 bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:3434 writesame 0xdb16f55 thru 0xdb1f277 data_size 0x1(0x8323 bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:3435 write 0x115ed12 thru 0x116e026 (0xf315 bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:3438 writesame 0x4892af5 thru 0x4893ed4 data_size 0x18(0x13e0 bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:3439 read 0xb034ff0 thru 0xb03772e (0x273f bytes) 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.423 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.435 INFO:teuthology.orchestra.run.smithi107.stdout:3444 trunc from 0xdb1f278 to 0x27992aa 2024-09-21T15:34:02.475 INFO:teuthology.orchestra.run.smithi107.stdout:3445 write 0x2a462cf thru 0x2a55aa2 (0xf7d4 bytes) 2024-09-21T15:34:02.482 INFO:teuthology.orchestra.run.smithi107.stdout:3446 read 0x1c853c thru 0x1c91b9 (0xc7e bytes) 2024-09-21T15:34:02.483 INFO:teuthology.orchestra.run.smithi107.stdout:3447 punch from 0x296e95e to 0x297316d, (0x480f bytes) 2024-09-21T15:34:02.521 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:02.520+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:02.525 INFO:teuthology.orchestra.run.smithi107.stdout:3449 writesame 0xecdf36b thru 0xecdfccc data_size 0x1(0x962 bytes) 2024-09-21T15:34:02.526 INFO:teuthology.orchestra.run.smithi107.stdout:3451 punch from 0xb1588d to 0xb16eac, (0x161f bytes) 2024-09-21T15:34:02.526 INFO:teuthology.orchestra.run.smithi107.stdout:3452 write 0xeb3f39f thru 0xeb4eeac (0xfb0e bytes) 2024-09-21T15:34:02.535 INFO:teuthology.orchestra.run.smithi107.stdout:3453 writesame 0xa7f51ea thru 0xa7f80dc data_size 0x11(0x2ef3 bytes) 2024-09-21T15:34:02.572 INFO:teuthology.orchestra.run.smithi107.stdout:3455 trunc from 0xecdfccd to 0x38185fb 2024-09-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:02.599 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:02.613 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:02.624 INFO:teuthology.orchestra.run.smithi107.stdout:3457 write 0x80c0609 thru 0x80c284e (0x2246 bytes) 2024-09-21T15:34:02.647 INFO:teuthology.orchestra.run.smithi107.stdout:3458 writesame 0x9d59fd0 thru 0x9d67b19 data_size 0x1(0xdb4a bytes) 2024-09-21T15:34:02.647 INFO:teuthology.orchestra.run.smithi107.stdout:3459 punch from 0x14c763e to 0x14d6407, (0xedc9 bytes) 2024-09-21T15:34:02.647 INFO:teuthology.orchestra.run.smithi107.stdout:3460 read 0x3c6651c thru 0x3c7369f (0xd184 bytes) 2024-09-21T15:34:02.657 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:02.656+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:02.659 INFO:teuthology.orchestra.run.smithi107.stdout:3461 write 0xa707e3d thru 0xa71272e (0xa8f2 bytes) 2024-09-21T15:34:02.669 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.669 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.669 INFO:teuthology.orchestra.run.smithi107.stdout:3465 writesame 0x700554e thru 0x700691a data_size 0x1(0x13cd bytes) 2024-09-21T15:34:02.686 INFO:teuthology.orchestra.run.smithi107.stdout:3466 trunc from 0xa71272f to 0xeaab34d 2024-09-21T15:34:02.696 INFO:teuthology.orchestra.run.smithi107.stdout:3467 write 0xaf6cdea thru 0xaf6daf1 (0xd08 bytes) 2024-09-21T15:34:02.701 INFO:teuthology.orchestra.run.smithi107.stdout:3468 read 0x54a38c7 thru 0x54ac54e (0x8c88 bytes) 2024-09-21T15:34:02.702 INFO:teuthology.orchestra.run.smithi107.stdout:3470 read 0xb873fa4 thru 0xb8836fa (0xf757 bytes) 2024-09-21T15:34:02.702 INFO:teuthology.orchestra.run.smithi107.stdout:3473 trunc from 0xeaab34d to 0xed7b179 2024-09-21T15:34:02.712 INFO:teuthology.orchestra.run.smithi107.stdout:3474 write 0x2609262 thru 0x2610ef7 (0x7c96 bytes) 2024-09-21T15:34:02.720 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.720 INFO:teuthology.orchestra.run.smithi107.stdout:3476 writesame 0xa31e24a thru 0xa31e86b data_size 0x1(0x622 bytes) 2024-09-21T15:34:02.722 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:02.720+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:3479 punch from 0xa7008da to 0xa70d4b5, (0xcbdb bytes) 2024-09-21T15:34:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:3481 punch from 0x33125fa to 0x3319621, (0x7027 bytes) 2024-09-21T15:34:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:3482 writesame 0x65ea58 thru 0x66afb4 data_size 0x2f(0xc55d bytes) 2024-09-21T15:34:02.730 INFO:teuthology.orchestra.run.smithi107.stdout:3483 write 0xc004ecd thru 0xc012b48 (0xdc7c bytes) 2024-09-21T15:34:02.737 INFO:teuthology.orchestra.run.smithi107.stdout:3484 read 0x29ad606 thru 0x29af09a (0x1a95 bytes) 2024-09-21T15:34:02.737 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.738 INFO:teuthology.orchestra.run.smithi107.stdout:3486 read 0xf2b29a thru 0xf37b3d (0xc8a4 bytes) 2024-09-21T15:34:02.738 INFO:teuthology.orchestra.run.smithi107.stdout:3487 write 0x256095d thru 0x25632cc (0x2970 bytes) 2024-09-21T15:34:02.742 INFO:teuthology.orchestra.run.smithi107.stdout:3488 read 0x1d811bf thru 0x1d824cd (0x130f bytes) 2024-09-21T15:34:02.742 INFO:teuthology.orchestra.run.smithi107.stdout:3490 read 0x1392d76 thru 0x139bb49 (0x8dd4 bytes) 2024-09-21T15:34:02.743 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.743 INFO:teuthology.orchestra.run.smithi107.stdout:3493 write 0x5c9d5ad thru 0x5ca8e78 (0xb8cc bytes) 2024-09-21T15:34:02.750 INFO:teuthology.orchestra.run.smithi107.stdout:3494 writesame 0x180a586 thru 0x180e2fb data_size 0x1(0x3d76 bytes) 2024-09-21T15:34:02.771 INFO:teuthology.orchestra.run.smithi107.stdout:3496 write 0x116541c thru 0x11751e2 (0xfdc7 bytes) 2024-09-21T15:34:02.781 INFO:teuthology.orchestra.run.smithi107.stdout:3497 write 0xed9322 thru 0xee5bbf (0xc89e bytes) 2024-09-21T15:34:02.787 INFO:teuthology.orchestra.run.smithi107.stdout:3498 write 0xba490f thru 0xba9fd6 (0x56c8 bytes) 2024-09-21T15:34:02.791 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:02.791 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:02.791 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:02.793 INFO:teuthology.orchestra.run.smithi107.stdout:3499 writesame 0xdfdd658 thru 0xdfe2ead data_size 0x3(0x5856 bytes) 2024-09-21T15:34:02.806 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:02.824 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:02.819+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:02.831 INFO:teuthology.orchestra.run.smithi107.stdout:3500 trunc from 0xed7b179 to 0x5731e16 2024-09-21T15:34:02.854 INFO:teuthology.orchestra.run.smithi107.stdout:3501 read 0x1db1fd6 thru 0x1db3623 (0x164e bytes) 2024-09-21T15:34:02.854 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.876 INFO:teuthology.orchestra.run.smithi107.stdout:3503 trunc from 0x5731e16 to 0xdee10a0 2024-09-21T15:34:02.886 INFO:teuthology.orchestra.run.smithi107.stdout:3504 read 0x46fa9d3 thru 0x47063a6 (0xb9d4 bytes) 2024-09-21T15:34:02.886 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.916 INFO:teuthology.orchestra.run.smithi107.stdout:3506 trunc from 0xdee10a0 to 0x107803f 2024-09-21T15:34:02.924 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:02.923+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:02.938 INFO:teuthology.orchestra.run.smithi107.stdout:3508 punch from 0x6ae17e to 0x6b72f7, (0x9179 bytes) 2024-09-21T15:34:02.938 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.938 INFO:teuthology.orchestra.run.smithi107.stdout:3510 read 0x1ea38f thru 0x1f6677 (0xc2e9 bytes) 2024-09-21T15:34:02.939 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:02.963 INFO:teuthology.orchestra.run.smithi107.stdout:3512 write 0x7c747a4 thru 0x7c82eb4 (0xe711 bytes) 2024-09-21T15:34:02.972 INFO:teuthology.orchestra.run.smithi107.stdout:3514 read 0x5cb667 thru 0x5d1d7a (0x6714 bytes) 2024-09-21T15:34:02.973 INFO:teuthology.orchestra.run.smithi107.stdout:3515 read 0x41ea4e3 thru 0x41f4b24 (0xa642 bytes) 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.2000106860000006, 2024-09-21T15:34:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.199632695, 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1994134400000007, 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1990162770000001, 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1986124890000003, 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1981567270000006, 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1909301810000006, 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1905638930000002, 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.012 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:03.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:03.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1898520500000007, 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.016 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1865915030000007, 2024-09-21T15:34:03.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.024 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1861512839999993, 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:03.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:03.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.028 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1858398609999998, 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:03.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:03.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:03.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.032 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1609122529999993, 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:03.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:03.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1602825970000001, 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:03.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1598781339999995, 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:03.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:03.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:03.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1593397410000001, 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.045 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:03.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:03.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:03.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1588127230000005, 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.049 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1583909979999998, 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:03.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:03.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.157975596, 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:03.057 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:03.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:03.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:03.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 8.1574251689999997, 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:03.061 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:03.065 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:03.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:34:03.068 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:03.064+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:03.068 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:03.068 INFO:teuthology.orchestra.run.smithi107.stdout:3516 writesame 0x86e49fa thru 0x86ee11c data_size 0x9(0x9723 bytes) 2024-09-21T15:34:03.068 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3520 write 0x91fc48d thru 0x920200d (0x5b81 bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3521 punch from 0xc6a1a5 to 0xc7a0d1, (0xff2c bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3523 writesame 0x6077253 thru 0x6085170 data_size 0x1(0xdf1e bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3524 writesame 0xc92bdb4 thru 0xc939420 data_size 0x1(0xd66d bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3525 write 0x77fae82 thru 0x77fc32a (0x14a9 bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3526 punch from 0x4a0187a to 0x4a05ee1, (0x4667 bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3527 read 0x8cc114 thru 0x8d5f46 (0x9e33 bytes) 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:03.069 INFO:teuthology.orchestra.run.smithi107.stdout:3532 read 0x33e460d thru 0x33e7e79 (0x386d bytes) 2024-09-21T15:34:03.096 INFO:teuthology.orchestra.run.smithi107.stdout:3533 trunc from 0xc939421 to 0x401fef 2024-09-21T15:34:03.127 INFO:teuthology.orchestra.run.smithi107.stdout:3535 read 0x398ced thru 0x39ff6d (0x7281 bytes) 2024-09-21T15:34:03.168 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:03.164+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:03.169 INFO:teuthology.orchestra.run.smithi107.stdout:3536 write 0xbf1e242 thru 0xbf241bb (0x5f7a bytes) 2024-09-21T15:34:03.178 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:03.178 INFO:teuthology.orchestra.run.smithi107.stdout:3538 punch from 0x589020b to 0x589b921, (0xb716 bytes) 2024-09-21T15:34:03.178 INFO:teuthology.orchestra.run.smithi107.stdout:3539 read 0x177a9de thru 0x1786f2e (0xc551 bytes) 2024-09-21T15:34:03.178 INFO:teuthology.orchestra.run.smithi107.stdout:3541 punch from 0x6f3d82f to 0x6f4d6b1, (0xfe82 bytes) 2024-09-21T15:34:03.178 INFO:teuthology.orchestra.run.smithi107.stdout:3542 clone 26 order 22 su 524288 sc 5 2024-09-21T15:34:03.200 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:03.200 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:03.201 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:03.201 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:03.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:03.269 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:03.268+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:03.370 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:03.368+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:03.378 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:03.378 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:03.378 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:03.392 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:03.471 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:03.468+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.516 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 64.803702736999995, 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:03.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:34:03.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:03.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:4211 3.18 e31/21 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head v 31'45, mlcod=30'43)", 2024-09-21T15:34:03.522 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 64.793380442, 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.032177192, 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719990+0000", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.719987+0000", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727828+0000", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0078418189999999999 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727854+0000", 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5429999999999999e-05 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727913+0000", 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9689999999999999e-05 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727972+0000", 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8488e-05 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.727991+0000", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9080999999999999e-05 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752088+0000", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024096999000000001 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752167+0000", 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8941000000000002e-05 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.865883216999997, 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:34:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.532 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:5402 3.12 e33/21 ondisk, result = 0)", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.831971783, 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024309259999999999, 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681398+0000", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681397+0000", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681405+0000", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5650000000000004e-06 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681427+0000", 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2303e-05 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.681438+0000", 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0856000000000001e-05 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705592+0000", 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.024154143999999999 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705602+0000", 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0417e-05 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705708+0000", 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000105356 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:10948 3.1 3:82fda950:::rbd_data.11917b2c111e.0000000000000011:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-21T15:34:03.536 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 50.832009053, 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025101894999999999, 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 10948 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681361+0000", 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681359+0000", 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681370+0000", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.135e-05 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681391+0000", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0633e-05 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681400+0000", 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8109999999999992e-06 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681492+0000", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2101000000000005e-05 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.681709+0000", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021704800000000001 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.687063+0000", 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053539759999999999 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706359+0000", 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019296002 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706374+0000", 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5245999999999999e-05 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706424+0000", 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9996e-05 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:12.706463+0000", 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9069999999999997e-05 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11973 3.18 3:1bdac4bd:::rbd_data.11917b2c111e.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:03.542 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 48.412041780999999, 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.019588250000000001, 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11973 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101328+0000", 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101326+0000", 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101338+0000", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2381000000000001e-05 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101361+0000", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2884e-05 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101372+0000", 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0716999999999999e-05 2024-09-21T15:34:03.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101427+0000", 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4626000000000002e-05 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.101493+0000", 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6115e-05 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:34:03.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.104640+0000", 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003147501 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120784+0000", 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016143372999999999 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120799+0000", 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5832999999999999e-05 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120871+0000", 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1977000000000006e-05 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.120917+0000", 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5269999999999999e-05 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.548 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16569 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524771255000001, 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016416218, 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16569 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988599+0000", 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988600+0000", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3519999999999999e-06 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988603+0000", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4070000000000002e-06 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988606+0000", 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7350000000000001e-06 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004678+0000", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016071621000000001 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.004772+0000", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4575999999999997e-05 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005015+0000", 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000242752 2024-09-21T15:34:03.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16570 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524761823999999, 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016891402999999999, 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16570 2024-09-21T15:34:03.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988608+0000", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988609+0000", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3659999999999999e-06 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988612+0000", 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0070000000000002e-06 2024-09-21T15:34:03.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988615+0000", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5160000000000001e-06 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005071+0000", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016456487999999998 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005205+0000", 2024-09-21T15:34:03.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000133638 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005500+0000", 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029470199999999998 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16571 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524753256, 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017340120000000001, 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16571 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988617+0000", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988618+0000", 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2559999999999999e-06 2024-09-21T15:34:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988622+0000", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.087e-06 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988626+0000", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1559999999999997e-06 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005524+0000", 2024-09-21T15:34:03.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016897760000000001 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005594+0000", 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9998000000000003e-05 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005957+0000", 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036308 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16572 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524740361999999, 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017681184999999999, 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16572 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988630+0000", 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988632+0000", 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.227e-06 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988638+0000", 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1990000000000003e-06 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988641+0000", 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.835e-06 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005986+0000", 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017345098999999999 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006062+0000", 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5685999999999993e-05 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006311+0000", 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000249452 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16574 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524727133999999, 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956159999999999, 2024-09-21T15:34:03.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16574 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988643+0000", 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988644+0000", 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.35e-06 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988647+0000", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1750000000000001e-06 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988650+0000", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.553e-06 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006336+0000", 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017685671 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006376+0000", 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0154999999999998e-05 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006599+0000", 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022348800000000001 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.570 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16575 3.6 3:6444445e:::rbd_data.1191dfb05674.0000000000000002:16 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524716852000001, 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01820097, 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16575 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988653+0000", 2024-09-21T15:34:03.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988654+0000", 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1930000000000001e-06 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988657+0000", 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.016e-06 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.988661+0000", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2200000000000001e-06 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006625+0000", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017964285 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006669+0000", 2024-09-21T15:34:03.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4363000000000002e-05 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006854+0000", 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000185105 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16729 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524301661999999, 2024-09-21T15:34:03.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016513967000000001, 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16729 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989069+0000", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989068+0000", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989070+0000", 2024-09-21T15:34:03.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.809e-06 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989076+0000", 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0530000000000003e-06 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989083+0000", 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9450000000000004e-06 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005427+0000", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016344145000000001 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005495+0000", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8040999999999994e-05 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005583+0000", 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7250000000000007e-05 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16732 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524283936, 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016636597999999999, 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16732 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.580 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989086+0000", 2024-09-21T15:34:03.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989091+0000", 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4739999999999996e-06 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989097+0000", 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9909999999999999e-06 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989102+0000", 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.792e-06 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005607+0000", 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016505407 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005670+0000", 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2288999999999999e-05 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005723+0000", 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3154e-05 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16735 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524264248000001, 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016763145, 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16735 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989106+0000", 2024-09-21T15:34:03.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989105+0000", 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989108+0000", 2024-09-21T15:34:03.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7149999999999998e-06 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989123+0000", 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5291999999999999e-05 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989128+0000", 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9649999999999999e-06 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005749+0000", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016620945000000002 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005806+0000", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6436e-05 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005869+0000", 2024-09-21T15:34:03.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3303e-05 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16738 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.524237341999999, 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016893127000000001, 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16738 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989133+0000", 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989132+0000", 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989135+0000", 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4530000000000001e-06 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989141+0000", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0599999999999996e-06 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989146+0000", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005907+0000", 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016761426999999999 2024-09-21T15:34:03.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.005977+0000", 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0178000000000005e-05 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006026+0000", 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8862000000000001e-05 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:16740 3.b 3:d1e93d0c:::rbd_data.1191bd191516.0000000000000000:14 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 29.52422069, 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016975200999999999, 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 16740 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.594 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989150+0000", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989149+0000", 2024-09-21T15:34:03.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989151+0000", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2170000000000001e-06 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989157+0000", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.631e-06 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:33.989161+0000", 2024-09-21T15:34:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4700000000000004e-06 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006040+0000", 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016878955000000001 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006077+0000", 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6838999999999997e-05 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:34.006125+0000", 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7547999999999999e-05 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:103668 3.b 3:d0264c1b:::rbd_data.1191b30e35a1.000000000000002e:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0986226170000002, 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023214459999999999, 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 103668 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414748+0000", 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414746+0000", 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414758+0000", 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.255e-05 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414790+0000", 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1846e-05 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414807+0000", 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7054e-05 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414928+0000", 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120555 2024-09-21T15:34:03.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.415132+0000", 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020484499999999999 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 3", 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.419576+0000", 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044434260000000003 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437770+0000", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018194248 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437781+0000", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1147e-05 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437857+0000", 2024-09-21T15:34:03.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5519000000000007e-05 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.437962+0000", 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010523599999999999 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:103669 3.1b 3:dee5ef32:::rbd_data.1191b30e35a1.000000000000002f:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:34:03.604 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.0985181839999996, 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030556359000000002, 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 103669 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:34:03.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414852+0000", 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414851+0000", 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414858+0000", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3130000000000001e-06 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414872+0000", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3529999999999999e-05 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414881+0000", 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8899999999999996e-06 2024-09-21T15:34:03.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.414997+0000", 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116709 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.415171+0000", 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017392800000000001 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:03.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.421427+0000", 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062557539999999997 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445200+0000", 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.023773424000000001 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445223+0000", 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2442999999999998e-05 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445302+0000", 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9546e-05 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:58.445408+0000", 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010597199999999999 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:107467 3.6 3:61705867:::rbd_data.11914af46aca.0000000000000060:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:03.006852+0000", 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.50651795300000002, 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031508598999999998, 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 107467 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006852+0000", 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:03.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006852+0000", 2024-09-21T15:34:03.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:03.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006849+0000", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006867+0000", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7876e-05 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006904+0000", 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7552999999999998e-05 2024-09-21T15:34:03.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.006923+0000", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8326999999999999e-05 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.007066+0000", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000143265 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.007454+0000", 2024-09-21T15:34:03.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038849300000000001 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 1", 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.012581+0000", 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051269109999999996 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.038185+0000", 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.025603719 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.038209+0000", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4443e-05 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.038297+0000", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7309e-05 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:03.038361+0000", 2024-09-21T15:34:03.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4176000000000004e-05 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:03.618 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:03.619 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:03.620 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:03.615+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:03.720 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:03.716+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:03.788 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:03.788 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:03.788 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:03.788 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:03.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:03.820 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:03.820+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:03.921 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:03.920+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:03.923 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:03.923 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:03.923 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:03.923 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:03.924 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:03.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:03.989 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:34:04.022 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:04.019+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:04.064 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 48.940613063000001, 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:04.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:04.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:04.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:04.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:04.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.49845685, 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:04.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:04.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.498432103999999, 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:04.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.498387154, 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:04.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494530495999999, 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.085 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:04.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494486135999999, 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:04.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494432787000001, 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494401394, 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.098 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.101 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494355668000001, 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:04.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:04.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:04.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494326295, 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.106 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:04.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:04.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494301735000001, 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:04.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:04.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:04.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.494275097999999, 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:04.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490436531, 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.119 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:04.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490406710999999, 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:04.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490381181, 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:04.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.132 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490355475000001, 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:04.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490305435, 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.137 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:04.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:04.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490270635000002, 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.141 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:04.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:04.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:04.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.490245986000001, 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.486195579, 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:04.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:04.153 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:04.154 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:04.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:04.156 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:04.151+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:04.256 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:04.252+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T15:34:04.305 INFO:teuthology.orchestra.run.smithi107.stdout:3543 write 0xe5a375c thru 0xe5aa96a (0x720f bytes) 2024-09-21T15:34:04.312 INFO:teuthology.orchestra.run.smithi107.stdout:3544 read 0x74ce307 thru 0x74d0a68 (0x2762 bytes) 2024-09-21T15:34:04.312 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:04.312 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:04.319 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:04.319 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:04.319 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:04.319 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:04.328 INFO:teuthology.orchestra.run.smithi107.stdout:3550 trunc from 0xe5aa96b to 0x3e7a7da 2024-09-21T15:34:04.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:04.342 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:04.342 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:04.357 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:04.355+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:04.361 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:04.373 INFO:teuthology.orchestra.run.smithi107.stdout:3551 writesame 0x775d5f8 thru 0x776c548 data_size 0x1(0xef51 bytes) 2024-09-21T15:34:04.373 INFO:teuthology.orchestra.run.smithi107.stdout:3552 writesame 0xba5891 thru 0xbb279e data_size 0x11(0xcf0e bytes) 2024-09-21T15:34:04.380 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:04.380 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:04.394 INFO:teuthology.orchestra.run.smithi107.stdout:3553 trunc from 0x776c549 to 0x402d211 2024-09-21T15:34:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:04.397 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:04.404 INFO:teuthology.orchestra.run.smithi107.stdout:3556 read 0x1dedfc9 thru 0x1df8a90 (0xaac8 bytes) 2024-09-21T15:34:04.407 INFO:teuthology.orchestra.run.smithi107.stdout:3557 trunc from 0x402d211 to 0x59f7ac3 2024-09-21T15:34:04.419 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:04.419 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:04.422 INFO:teuthology.orchestra.run.smithi107.stdout:3558 trunc from 0x59f7ac3 to 0xa9d5992 2024-09-21T15:34:04.431 INFO:teuthology.orchestra.run.smithi107.stdout:3559 trunc from 0xa9d5992 to 0x8d37c1d 2024-09-21T15:34:04.442 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:04.446 INFO:teuthology.orchestra.run.smithi107.stdout:3560 writesame 0x987d8b3 thru 0x988cd53 data_size 0x19(0xf4a1 bytes) 2024-09-21T15:34:04.446 INFO:teuthology.orchestra.run.smithi107.stdout:3563 read 0x78f97 thru 0x7f68c (0x66f6 bytes) 2024-09-21T15:34:04.447 INFO:teuthology.orchestra.run.smithi107.stdout:3564 writesame 0x2f100e5 thru 0x2f17df3 data_size 0x13(0x7d0f bytes) 2024-09-21T15:34:04.453 INFO:teuthology.orchestra.run.smithi107.stdout:3567 read 0x3bb0735 thru 0x3bb97c5 (0x9091 bytes) 2024-09-21T15:34:04.454 INFO:teuthology.orchestra.run.smithi107.stdout:3568 trunc from 0x988cd54 to 0xa2354a4 2024-09-21T15:34:04.457 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:04.455+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:04.461 INFO:teuthology.orchestra.run.smithi107.stdout:3569 clone 27 order 18 su 2048 sc 5 2024-09-21T15:34:04.464 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:04.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:04.484 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:04.484 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:04.559 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:04.555+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:04.595 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:04.595 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:04.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:04.660 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:04.656+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:04.761 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:04.759+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:04.769 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:04.770 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:04.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:04.872 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:04.871+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:04.949 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:04.949 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:04.962 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:04.960+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:04.964 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:04.980 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:04.980 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:04.980 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.911282459542344 2024-09-21T15:34:04.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.911282459542344 2024-09-21T15:34:05.062 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:05.060+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:05.104 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone26 2024-09-21T15:34:05.131 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.131 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:05.197 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:05.196+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:05.264 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:05.260+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:05.280 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T15:34:05.284 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.284 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:05.365 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:05.363+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:05.465 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:05.463+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:05.470 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.470 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.484 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:05.566 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:05.564+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:05.607 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.607 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.609 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:05.609 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:05.610 INFO:teuthology.orchestra.run.smithi107.stdout:3574 punch from 0x51e6c39 to 0x51e8aac, (0x1e73 bytes) 2024-09-21T15:34:05.616 INFO:teuthology.orchestra.run.smithi107.stdout:3576 write 0x8f66214 thru 0x8f71cc8 (0xbab5 bytes) 2024-09-21T15:34:05.621 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:05.644 INFO:teuthology.orchestra.run.smithi107.stdout:3577 writesame 0xe5ed848 thru 0xe5f173f data_size 0x41(0x3ef8 bytes) 2024-09-21T15:34:05.654 INFO:teuthology.orchestra.run.smithi107.stdout:3578 trunc from 0xe5f1740 to 0x6657783 2024-09-21T15:34:05.666 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:05.663+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:05.682 INFO:teuthology.orchestra.run.smithi107.stdout:3579 writesame 0xf47758 thru 0xf558a6 data_size 0x1(0xe14f bytes) 2024-09-21T15:34:05.739 INFO:teuthology.orchestra.run.smithi107.stdout:3581 read 0x245d8d8 thru 0x246b77f (0xdea8 bytes) 2024-09-21T15:34:05.739 INFO:teuthology.orchestra.run.smithi107.stdout:3582 writesame 0x1043dc4 thru 0x104ca82 data_size 0x1(0x8cbf bytes) 2024-09-21T15:34:05.767 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:05.763+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:05.810 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.811 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.822 INFO:teuthology.orchestra.run.smithi107.stdout:3583 write 0xd36e947 thru 0xd36f04a (0x704 bytes) 2024-09-21T15:34:05.824 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:05.829 INFO:teuthology.orchestra.run.smithi107.stdout:3584 write 0x5a4a232 thru 0x5a53dd4 (0x9ba3 bytes) 2024-09-21T15:34:05.845 INFO:teuthology.orchestra.run.smithi107.stdout:3585 clone 28 order 23 su 262144 sc 5 2024-09-21T15:34:05.869 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:05.868+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:05.947 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:05.947 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:05.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:06.003 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.5 to 0.911282 (e949) 2024-09-21T15:34:06.009 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:06.008+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:06.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:34:06.070 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:06.068+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:06.072 INFO:teuthology.orchestra.run.smithi107.stdout:leaving image image_client.0-clone27 intact 2024-09-21T15:34:06.135 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:06.135 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:06.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:06.171 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:06.167+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:06.240 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T15:34:06.272 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:06.271+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:06.319 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:06.319 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:06.333 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:06.377 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:06.376+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:06.454 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:06.454 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:06.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:06.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:06.509 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:06.508+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:06.516 INFO:teuthology.orchestra.run.smithi107.stdout:3586 write 0xba8ba9 thru 0xbb308f (0xa4e7 bytes) 2024-09-21T15:34:06.574 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:06.571+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:06.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:06.658 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:06.674 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:06.675 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:06.671+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:06.679 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:06.680 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:06.690 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:06.690 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:06.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:06.713 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:06.730 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:06.746 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:06.763 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:06.775 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:06.771+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:06.781 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:06.781 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi107.stdout:3588 punch from 0xc1bf3a to 0xc1c276, (0x33c bytes) 2024-09-21T15:34:06.816 INFO:teuthology.orchestra.run.smithi107.stdout:3589 write 0x3f0097a thru 0x3f0e19e (0xd825 bytes) 2024-09-21T15:34:06.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:06.876 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:06.875+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:06.967 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:06.967 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:06.967 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:06.967 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:06.967 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:06.976 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:06.976+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:06.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:07.018 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:34:07.077 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:07.076+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:3593 write 0x5c18c5f thru 0x5c243ce (0xb770 bytes) 2024-09-21T15:34:07.159 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:07.159 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:07.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:07.159 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:07.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.380107735999999, 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:07.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.379729745000001, 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.164 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.168 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.379510489999999, 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:07.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.379113327000001, 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.378709539000001, 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.177 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:07.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:07.181 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.378253776999999, 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:07.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:07.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.371027230999999, 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.370660943000001, 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.190 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.369949099999999, 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.194 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.196 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.198 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.366688552999999, 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:07.199 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:07.200 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:07.201 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.202 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.366248334, 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.203 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:07.204 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.205 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.206 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.365936911, 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.207 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:07.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.210 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.211 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.341009303, 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:07.212 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:07.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:07.214 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.215 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.340379647000001, 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.216 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:07.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:07.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.219 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.339975184, 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.220 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:07.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:07.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.223 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.339436791000001, 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.224 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.225 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.226 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.338909772999999, 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.228 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.229 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.230 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:07.231 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.338488048, 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.233 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.234 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.235 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.236 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.338072646000001, 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:07.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:07.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:07.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:07.240 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 12.337522219, 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:07.241 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:07.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:07.245 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:07.246 INFO:teuthology.orchestra.run.smithi107.stdout:3594 write 0x74a4ab4 thru 0x74a6a96 (0x1fe3 bytes) 2024-09-21T15:34:07.246 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:07.246 INFO:teuthology.orchestra.run.smithi107.stdout:3597 punch from 0xfe549f to 0xfeeecc, (0x9a2d bytes) 2024-09-21T15:34:07.247 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:07.244+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:07.247 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:07.248 INFO:teuthology.orchestra.run.smithi107.stdout:3598 trunc from 0xd36f04b to 0x34db49d 2024-09-21T15:34:07.347 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:07.344+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:07.372 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:07.372 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:07.372 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:07.373 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:07.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:07.448 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:07.444+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:07.541 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:07.541 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:07.549 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:07.548+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:07.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:07.650 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:07.648+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:07.735 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:07.735 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:07.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 69.024118892999994, 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.736 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:34:07.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.086299373000003, 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:34:07.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:34:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:34:07.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112367 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1921024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.850493+0000", 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88329377499999995, 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037635353000000003, 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.748 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112367 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850493+0000", 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850493+0000", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850492+0000", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850495+0000", 2024-09-21T15:34:07.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.174e-06 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850504+0000", 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7690000000000002e-06 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850510+0000", 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0850000000000002e-06 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.887990+0000", 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037479726999999997 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888059+0000", 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9215999999999996e-05 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888128+0000", 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9102000000000003e-05 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112369 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1931264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.850746+0000", 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88304017700000004, 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037678898000000002, 2024-09-21T15:34:07.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112369 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850746+0000", 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850746+0000", 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850745+0000", 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850748+0000", 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.915e-06 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850756+0000", 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4569999999999999e-06 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.850762+0000", 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1229999999999997e-06 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888268+0000", 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037505961999999997 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888356+0000", 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8487999999999997e-05 2024-09-21T15:34:07.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888425+0000", 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8668000000000001e-05 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112371 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.758 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.851038+0000", 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88274862799999998, 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037671705999999999, 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112371 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851038+0000", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851038+0000", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851037+0000", 2024-09-21T15:34:07.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851040+0000", 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1599999999999998e-06 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851048+0000", 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8790000000000003e-06 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851054+0000", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0399999999999998e-06 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888554+0000", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037499688000000003 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888640+0000", 2024-09-21T15:34:07.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6113999999999996e-05 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888709+0000", 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9505000000000006e-05 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112373 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1951744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.851277+0000", 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88250904200000002, 2024-09-21T15:34:07.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037752260000000003, 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112373 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851277+0000", 2024-09-21T15:34:07.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851277+0000", 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851277+0000", 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:07.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851280+0000", 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0319999999999999e-06 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851287+0000", 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4680000000000003e-06 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851293+0000", 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9599999999999997e-06 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888864+0000", 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037570441000000003 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.767 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.888941+0000", 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7795999999999999e-05 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889030+0000", 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8269000000000005e-05 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.768 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112375 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.851550+0000", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88223649299999995, 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037683859, 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112375 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851550+0000", 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851550+0000", 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851549+0000", 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851552+0000", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2940000000000001e-06 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851563+0000", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0516999999999999e-05 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851569+0000", 2024-09-21T15:34:07.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2430000000000002e-06 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889151+0000", 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037581437000000002 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889198+0000", 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7027999999999998e-05 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889234+0000", 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6062999999999997e-05 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112376 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1953792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.851760+0000", 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88202641199999998, 2024-09-21T15:34:07.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037666975999999998, 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112376 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851760+0000", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851760+0000", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851574+0000", 2024-09-21T15:34:07.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999814 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851764+0000", 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000190373 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851780+0000", 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5311e-05 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851791+0000", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1151e-05 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889264+0000", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037472921999999999 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889337+0000", 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2945e-05 2024-09-21T15:34:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889427+0000", 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0280999999999998e-05 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112377 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1972224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.851799+0000", 2024-09-21T15:34:07.778 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88198698099999995, 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037734979000000002, 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112377 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.779 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851799+0000", 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851799+0000", 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851799+0000", 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:07.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851808+0000", 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8799999999999997e-06 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851815+0000", 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8399999999999995e-06 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.851821+0000", 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0460000000000002e-06 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889459+0000", 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037637821000000002 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889497+0000", 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7347e-05 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889534+0000", 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7753999999999998e-05 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112379 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1982464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.852074+0000", 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88171201499999996, 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037739966, 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112379 2024-09-21T15:34:07.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852074+0000", 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852074+0000", 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852074+0000", 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852077+0000", 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3079999999999999e-06 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852087+0000", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.022e-05 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852093+0000", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1090000000000003e-06 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889669+0000", 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037576015999999997 2024-09-21T15:34:07.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889742+0000", 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3184999999999998e-05 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889814+0000", 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1962e-05 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112380 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1974272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.852204+0000", 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88158286799999996, 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037707638000000002, 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112380 2024-09-21T15:34:07.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852204+0000", 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852204+0000", 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852098+0000", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9998941 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852208+0000", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011008199999999999 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852219+0000", 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1506e-05 2024-09-21T15:34:07.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852227+0000", 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3540000000000002e-06 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889839+0000", 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037612120999999998 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889876+0000", 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7311999999999998e-05 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.889911+0000", 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4962000000000002e-05 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112382 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 1992704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.852530+0000", 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.88125613899999999, 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037644882999999997, 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112382 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.794 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852530+0000", 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852530+0000", 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852529+0000", 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852533+0000", 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9149999999999998e-06 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852541+0000", 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6739999999999997e-06 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852547+0000", 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0109999999999997e-06 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890104+0000", 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037557410999999999 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890140+0000", 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5423999999999998e-05 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890175+0000", 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5237000000000002e-05 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.798 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:112385 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 2004992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.852951+0000", 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.880835017, 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037705750000000003, 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 112385 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.799 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852951+0000", 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852951+0000", 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852951+0000", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852954+0000", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7860000000000001e-06 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852961+0000", 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3080000000000002e-06 2024-09-21T15:34:07.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.852967+0000", 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8459999999999996e-06 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890500+0000", 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037533284 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890571+0000", 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1077999999999999e-05 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.890657+0000", 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5970999999999994e-05 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.803 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117496 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.584355+0000", 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.149431709, 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037661662999999998, 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117496 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584355+0000", 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584355+0000", 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584251+0000", 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9998965 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584358+0000", 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000106883 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584367+0000", 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:07.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584373+0000", 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1149999999999999e-06 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.621907+0000", 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037533865999999999 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.621948+0000", 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1931000000000002e-05 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622016+0000", 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7854000000000005e-05 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.808 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117497 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.584456+0000", 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.149329922, 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037714485999999998, 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117497 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584456+0000", 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584456+0000", 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584383+0000", 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999261 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584460+0000", 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7065999999999997e-05 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584468+0000", 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4200000000000007e-06 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584481+0000", 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2858e-05 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622043+0000", 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037562208 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622104+0000", 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1150999999999998e-05 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622171+0000", 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6501999999999994e-05 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117498 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3811328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.584529+0000", 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.149256953, 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037772145999999999, 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117498 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.814 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584529+0000", 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584529+0000", 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584486+0000", 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999566 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584533+0000", 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6650000000000002e-05 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584541+0000", 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1880000000000002e-06 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584557+0000", 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6116999999999998e-05 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622198+0000", 2024-09-21T15:34:07.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037641301000000002 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622249+0000", 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1175999999999999e-05 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622302+0000", 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2132999999999997e-05 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.818 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117499 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3721216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.584758+0000", 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.149028093, 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037704199000000001, 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117499 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584758+0000", 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584758+0000", 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584562+0000", 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999804 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584768+0000", 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00020572599999999999 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584777+0000", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7909999999999993e-06 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584783+0000", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2570000000000004e-06 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622326+0000", 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037543281999999997 2024-09-21T15:34:07.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622395+0000", 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8233999999999998e-05 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622462+0000", 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7762000000000006e-05 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117500 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3821568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.584871+0000", 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.14891489999999999, 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037717168000000002, 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117500 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584871+0000", 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584871+0000", 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584788+0000", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999166 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584875+0000", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6774999999999995e-05 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584884+0000", 2024-09-21T15:34:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1910000000000006e-06 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584897+0000", 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2982999999999999e-05 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622495+0000", 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037597994000000003 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622541+0000", 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5946999999999999e-05 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622589+0000", 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7524000000000001e-05 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117501 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3731456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.585096+0000", 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.148689974, 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037642538000000003, 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117501 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585096+0000", 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585096+0000", 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.584903+0000", 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9998064 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585100+0000", 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019755700000000001 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585109+0000", 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5280000000000003e-06 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585115+0000", 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5479999999999998e-06 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622621+0000", 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037505668999999998 2024-09-21T15:34:07.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622682+0000", 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0770999999999998e-05 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.622739+0000", 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7274000000000001e-05 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:117504 3.1b 3:d9577442:::rbd_data.11914be32c41.0000000000000012:1e [read 3842048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.585593+0000", 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.148193203, 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037633847999999998, 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:07.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 117504 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585593+0000", 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585593+0000", 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585483+0000", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9998899 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585597+0000", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113621 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585605+0000", 2024-09-21T15:34:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3850000000000002e-06 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.585619+0000", 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.419e-05 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.623079+0000", 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037459665000000003 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.623155+0000", 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6484999999999997e-05 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.623227+0000", 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1701999999999999e-05 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:07.839 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:07.839 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:07.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:07.841 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:07.836+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:07.941 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:07.939+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:07.985 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:07.986 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:07.986 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:08.044 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:08.043+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:08.143 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:08.140+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:08.178 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:08.179 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:08.179 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:08.179 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:08.179 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:08.244 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:08.239+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:08.269 INFO:teuthology.orchestra.run.smithi107.stdout:3599 write 0x14dbe22 thru 0x14e3d31 (0x7f10 bytes) 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:08.332 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 53.207999729999997, 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:08.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:08.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:08.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:08.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.765843517, 2024-09-21T15:34:08.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:08.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.765818770999999, 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:08.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:08.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.765773821, 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:08.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761917163, 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:08.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:08.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:08.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761872802999999, 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761819454000001, 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.361 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:08.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:08.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:08.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761788061000001, 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:08.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:08.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:08.369 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761742335000001, 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:08.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:08.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761712962000001, 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:08.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761688402000001, 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.382 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.761661764999999, 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:08.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:08.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:08.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757823198000001, 2024-09-21T15:34:08.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:08.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757793377999999, 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.393 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:08.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:08.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:08.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757767848, 2024-09-21T15:34:08.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:08.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:08.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757742142000001, 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.402 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:08.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757692102, 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:08.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:08.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757657301999998, 2024-09-21T15:34:08.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.757632653000002, 2024-09-21T15:34:08.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:08.418 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.753582246000001, 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.420 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.422 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:08.424 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:08.420+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:08.524 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:08.520+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:08.555 INFO:teuthology.orchestra.run.smithi107.stdout:3601 trunc from 0x34db49d to 0x6638c56 2024-09-21T15:34:08.568 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:08.568 INFO:teuthology.orchestra.run.smithi107.stdout:3603 punch from 0x324c56d to 0x324d55b, (0xfee bytes) 2024-09-21T15:34:08.590 INFO:teuthology.orchestra.run.smithi107.stdout:3604 writesame 0xafa9cf3 thru 0xafb06d9 data_size 0x7(0x69e7 bytes) 2024-09-21T15:34:08.590 INFO:teuthology.orchestra.run.smithi107.stdout:3609 write 0x3b5ba52 thru 0x3b6950a (0xdab9 bytes) 2024-09-21T15:34:08.592 INFO:teuthology.orchestra.run.smithi107.stdout:3610 punch from 0x1e85a72 to 0x1e887db, (0x2d69 bytes) 2024-09-21T15:34:08.593 INFO:teuthology.orchestra.run.smithi107.stdout:3611 writesame 0x3e83194 thru 0x3e8e9bc data_size 0x23(0xb829 bytes) 2024-09-21T15:34:08.596 INFO:teuthology.orchestra.run.smithi107.stdout:3612 writesame 0x227a11a thru 0x228345e data_size 0x47(0x9345 bytes) 2024-09-21T15:34:08.605 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:08.606 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:08.606 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:08.606 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:08.625 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:08.624+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:08.726 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:08.724+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:08.749 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:08.749 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:08.749 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:08.749 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:08.749 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.762 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:08.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:08.827 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:08.824+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.332540989999998, 2024-09-21T15:34:08.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:34:08.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.861624719000005, 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:34:08.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.886 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.851834627000002, 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:08.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.891 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.837880587000001, 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:34:08.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:34:08.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.829713441999999, 2024-09-21T15:34:08.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:34:08.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.828967043000006, 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:08.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:08.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.810576067, 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:34:08.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.749710127, 2024-09-21T15:34:08.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 71.738039791000006, 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:34:08.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.779564692999998, 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:34:08.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:34:08.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.778653837, 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.925 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:08.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:08.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.777687909999997, 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:34:08.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:08.934 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.767483927999997, 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:08.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:08.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:15192 3.0 e58/50 3:002446f9:::rbd_data.11919eb7aeb1.000000000000004f:head v 58'302, mlcod=58'298)", 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.987769581000002, 2024-09-21T15:34:08.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032278039999999999, 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888712+0000", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888728+0000", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5973000000000001e-05 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888739+0000", 2024-09-21T15:34:08.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1039999999999999e-05 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888745+0000", 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6049999999999997e-06 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888780+0000", 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5373999999999997e-05 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888800+0000", 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9565e-05 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891899+0000", 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030989469999999999 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891941+0000", 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2187000000000001e-05 2024-09-21T15:34:08.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62123 3.4 e68/47 3:22ecafc8:::rbd_data.1191e6bca1cd.00000000000000a8:head v 68'99, mlcod=65'97)", 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.284152557999999, 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032546599999999999, 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592327+0000", 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592348+0000", 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1233000000000002e-05 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592397+0000", 2024-09-21T15:34:08.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8353000000000003e-05 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592417+0000", 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0653e-05 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592769+0000", 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035143300000000001 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592822+0000", 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3137000000000003e-05 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595483+0000", 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026611849999999999 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595585+0000", 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101462 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.948 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62269 3.0 e68/50 3:072ab387:::rbd_header.1191e6bca1cd:head v 68'364, mlcod=68'362)", 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.997942232, 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037514879999999999, 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878538+0000", 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878557+0000", 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9173000000000001e-05 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878592+0000", 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5429000000000002e-05 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878610+0000", 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7974000000000001e-05 2024-09-21T15:34:08.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878698+0000", 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7293000000000006e-05 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878730+0000", 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2175000000000003e-05 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882213+0000", 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034834689999999999 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882292+0000", 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8453000000000003e-05 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62282 3.0 e69/50 3:01c4b5a3:::rbd_data.1191e6bca1cd.00000000000000da:head v 69'365, mlcod=68'363)", 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:08.953 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.973816943999999, 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037099500000000001, 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902665+0000", 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902697+0000", 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1680000000000002e-05 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902716+0000", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9145999999999999e-05 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902726+0000", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0139000000000001e-05 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902782+0000", 2024-09-21T15:34:08.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5986999999999998e-05 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902801+0000", 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8300000000000001e-05 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906328+0000", 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035275829999999999 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906376+0000", 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:102949 3.4 e72/47 3:221b15ed:::rbd_data.1191b30e35a1.0000000000000042:head v 72'106, mlcod=72'104)", 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.53990361, 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040397410000000003, 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.958 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336577+0000", 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336598+0000", 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1659e-05 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336622+0000", 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3204e-05 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336633+0000", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0967e-05 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336686+0000", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3915999999999997e-05 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336712+0000", 2024-09-21T15:34:08.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5060999999999999e-05 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340579+0000", 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038670309999999999 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340619+0000", 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0080999999999998e-05 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.962 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:103104 3.9 e72/47 3:92ad05bc:::rbd_data.1191b30e35a1.000000000000002d:head v 72'107, mlcod=72'105)", 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.398393244999999, 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034052539999999999, 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478088+0000", 2024-09-21T15:34:08.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478140+0000", 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2240000000000001e-05 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478160+0000", 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0176000000000001e-05 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478172+0000", 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2133999999999999e-05 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478283+0000", 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110418 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478348+0000", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4735999999999998e-05 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481393+0000", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030456789999999999 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481495+0000", 2024-09-21T15:34:08.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101309 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.35192263099999999, 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:08.968 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:08.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:08.972 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:08.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:08.974 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:08.975 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:08.975 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:08.975 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:08.976 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:08.976 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:08.976 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:08.972+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:08.984 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:09.005 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:09.009 INFO:teuthology.orchestra.run.smithi107.stdout:3613 write 0xb4219ef thru 0xb425860 (0x3e72 bytes) 2024-09-21T15:34:09.015 INFO:teuthology.orchestra.run.smithi107.stdout:3615 writesame 0x383cc30 thru 0x383d9bb data_size 0xc(0xd8c bytes) 2024-09-21T15:34:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:3616 punch from 0x100fe21 to 0x1017a78, (0x7c57 bytes) 2024-09-21T15:34:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:3617 read 0xb259bc1 thru 0xb265f8c (0xc3cc bytes) 2024-09-21T15:34:09.018 INFO:teuthology.orchestra.run.smithi107.stdout:3618 write 0x1aee3d3 thru 0x1af63c5 (0x7ff3 bytes) 2024-09-21T15:34:09.023 INFO:teuthology.orchestra.run.smithi107.stdout:3620 write 0x344e67c thru 0x3457205 (0x8b8a bytes) 2024-09-21T15:34:09.026 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:09.026 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:09.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:34:09.039 INFO:teuthology.orchestra.run.smithi107.stdout:3621 writesame 0xe6de6c2 thru 0xe6eab96 data_size 0x1(0xc4d5 bytes) 2024-09-21T15:34:09.040 INFO:teuthology.orchestra.run.smithi107.stdout:3623 punch from 0x60f903b to 0x60f94f1, (0x4b6 bytes) 2024-09-21T15:34:09.044 INFO:teuthology.orchestra.run.smithi107.stdout:3626 trunc from 0xe6eab97 to 0xae5f660 2024-09-21T15:34:09.047 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:09.047 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:09.069 INFO:teuthology.orchestra.run.smithi107.stdout:3628 write 0xd072275 thru 0xd082131 (0xfebd bytes) 2024-09-21T15:34:09.074 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:09.074 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:09.074 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:09.072+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:09.077 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:09.078 INFO:teuthology.orchestra.run.smithi107.stdout:3630 read 0x66b090d thru 0x66bba20 (0xb114 bytes) 2024-09-21T15:34:09.078 INFO:teuthology.orchestra.run.smithi107.stdout:3633 punch from 0xcd761fd to 0xcd7f2f5, (0x90f8 bytes) 2024-09-21T15:34:09.078 INFO:teuthology.orchestra.run.smithi107.stdout:3634 write 0x3ce754f thru 0x3cf69d5 (0xf487 bytes) 2024-09-21T15:34:09.080 INFO:teuthology.orchestra.run.smithi107.stdout:3635 read 0xa521df1 thru 0xa5257b8 (0x39c8 bytes) 2024-09-21T15:34:09.080 INFO:teuthology.orchestra.run.smithi107.stdout:3636 write 0x530d334 thru 0x5313e6b (0x6b38 bytes) 2024-09-21T15:34:09.086 INFO:teuthology.orchestra.run.smithi107.stdout:3637 write 0xa73caa1 thru 0xa73ccf8 (0x258 bytes) 2024-09-21T15:34:09.090 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:09.090 INFO:teuthology.orchestra.run.smithi107.stdout:3642 writesame 0x18f4559 thru 0x18f48fd data_size 0x1(0x3a5 bytes) 2024-09-21T15:34:09.097 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.097 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:09.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:09.200 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:09.199+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:09.276 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:09.275+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:09.281 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.281 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:09.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:09.377 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:09.376+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:09.448 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.448 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:09.461 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:09.505 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:09.504+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:09.579 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:09.575+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:09.595 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.595 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:09.611 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:09.631 INFO:teuthology.orchestra.run.smithi107.stdout:3643 trunc from 0xd082132 to 0x614d497 2024-09-21T15:34:09.651 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:09.651 INFO:teuthology.orchestra.run.smithi107.stdout:3645 punch from 0x513ae22 to 0x5141742, (0x6920 bytes) 2024-09-21T15:34:09.651 INFO:teuthology.orchestra.run.smithi107.stdout:3646 read 0x47a92af thru 0x47b519b (0xbeed bytes) 2024-09-21T15:34:09.652 INFO:teuthology.orchestra.run.smithi107.stdout:3648 write 0x43bd9bf thru 0x43beb98 (0x11da bytes) 2024-09-21T15:34:09.655 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:09.655 INFO:teuthology.orchestra.run.smithi107.stdout:3652 write 0x1d07b2 thru 0x1dabb6 (0xa405 bytes) 2024-09-21T15:34:09.669 INFO:teuthology.orchestra.run.smithi107.stdout:3656 trunc from 0x614d497 to 0xab04c8a 2024-09-21T15:34:09.679 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:09.675+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:09.695 INFO:teuthology.orchestra.run.smithi107.stdout:3657 writesame 0xcb43eda thru 0xcb47df8 data_size 0xd(0x3f1f bytes) 2024-09-21T15:34:09.695 INFO:teuthology.orchestra.run.smithi107.stdout:3659 read 0x94210e thru 0x942a11 (0x904 bytes) 2024-09-21T15:34:09.695 INFO:teuthology.orchestra.run.smithi107.stdout:3660 write 0xb2b3782 thru 0xb2c1401 (0xdc80 bytes) 2024-09-21T15:34:09.703 INFO:teuthology.orchestra.run.smithi107.stdout:3661 read 0x694fc01 thru 0x695b86f (0xbc6f bytes) 2024-09-21T15:34:09.703 INFO:teuthology.orchestra.run.smithi107.stdout:3662 write 0x9cbe23f thru 0x9cc2d19 (0x4adb bytes) 2024-09-21T15:34:09.712 INFO:teuthology.orchestra.run.smithi107.stdout:3663 writesame 0x801e147 thru 0x8025a13 data_size 0x5(0x78cd bytes) 2024-09-21T15:34:09.721 INFO:teuthology.orchestra.run.smithi107.stdout:3664 punch from 0x752516e to 0x7529f0e, (0x4da0 bytes) 2024-09-21T15:34:09.748 INFO:teuthology.orchestra.run.smithi107.stdout:3665 trunc from 0xcb47df9 to 0x25655fd 2024-09-21T15:34:09.780 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:09.780+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:09.799 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.799 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:09.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:09.881 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:09.879+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:09.982 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:09.980+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:09.993 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:09.993 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.007 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:10.043 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:34:10.093 INFO:teuthology.orchestra.run.smithi107.stdout:3666 writesame 0xb5614ad thru 0xb56dbac data_size 0x80(0xc700 bytes) 2024-09-21T15:34:10.093 INFO:teuthology.orchestra.run.smithi107.stdout:3668 writesame 0x99816d0 thru 0x998333b data_size 0x4(0x1c6c bytes) 2024-09-21T15:34:10.097 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:10.096+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:3669 trunc from 0xb56dbad to 0x4a799b9 2024-09-21T15:34:10.120 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.120 INFO:teuthology.orchestra.run.smithi107.stdout:3671 punch from 0x108c908 to 0x109064c, (0x3d44 bytes) 2024-09-21T15:34:10.145 INFO:teuthology.orchestra.run.smithi107.stdout:3672 writesame 0xc5791bf thru 0xc5841e1 data_size 0x1(0xb023 bytes) 2024-09-21T15:34:10.150 INFO:teuthology.orchestra.run.smithi107.stdout:3673 trunc from 0xc5841e2 to 0xeabd5ae 2024-09-21T15:34:10.160 INFO:teuthology.orchestra.run.smithi107.stdout:3674 writesame 0x3420fe7 thru 0x3423fee data_size 0x3a(0x3008 bytes) 2024-09-21T15:34:10.165 INFO:teuthology.orchestra.run.smithi107.stdout:3676 read 0x3bbdd61 thru 0x3bcd3e5 (0xf685 bytes) 2024-09-21T15:34:10.166 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.166 INFO:teuthology.orchestra.run.smithi107.stdout:3679 read 0xe20f87f thru 0xe214a15 (0x5197 bytes) 2024-09-21T15:34:10.166 INFO:teuthology.orchestra.run.smithi107.stdout:3680 write 0x9ea2e6d thru 0x9eb0982 (0xdb16 bytes) 2024-09-21T15:34:10.169 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:10.169 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.171 INFO:teuthology.orchestra.run.smithi107.stdout:3681 write 0xdfcffd4 thru 0xdfdc8fa (0xc927 bytes) 2024-09-21T15:34:10.177 INFO:teuthology.orchestra.run.smithi107.stdout:3682 writesame 0x1869200 thru 0x187430a data_size 0x1(0xb10b bytes) 2024-09-21T15:34:10.183 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:10.179+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:10.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:10.211 INFO:teuthology.orchestra.run.smithi107.stdout:3683 writesame 0x2aaff7b thru 0x2ab393a data_size 0x60(0x39c0 bytes) 2024-09-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.218 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.227 INFO:teuthology.orchestra.run.smithi107.stdout:3686 trunc from 0xeabd5ae to 0xb42f81d 2024-09-21T15:34:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:3688 read 0x3276f83 thru 0x32787f8 (0x1876 bytes) 2024-09-21T15:34:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:3691 write 0x8005c13 thru 0x800d536 (0x7924 bytes) 2024-09-21T15:34:10.252 INFO:teuthology.orchestra.run.smithi107.stdout:3697 writesame 0x78f5327 thru 0x7901fb2 data_size 0x26(0xcc8c bytes) 2024-09-21T15:34:10.284 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:10.283+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:10.286 INFO:teuthology.orchestra.run.smithi107.stdout:3698 trunc from 0xb42f81d to 0x148fabe 2024-09-21T15:34:10.332 INFO:teuthology.orchestra.run.smithi107.stdout:3699 writesame 0x53a7478 thru 0x53b6edc data_size 0x17(0xfa65 bytes) 2024-09-21T15:34:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:3700 writesame 0x733852c thru 0x7340696 data_size 0x1(0x816b bytes) 2024-09-21T15:34:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:3701 writesame 0x3d59684 thru 0x3d5e3d2 data_size 0x9(0x4d4f bytes) 2024-09-21T15:34:10.362 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:10.362 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.376 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:10.382 INFO:teuthology.orchestra.run.smithi107.stdout:3702 write 0xc2e4793 thru 0xc2eafac (0x681a bytes) 2024-09-21T15:34:10.389 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.389 INFO:teuthology.orchestra.run.smithi107.stdout:3704 writesame 0x65faded thru 0x6608a3c data_size 0xc(0xdc50 bytes) 2024-09-21T15:34:10.399 INFO:teuthology.orchestra.run.smithi107.stdout:3706 write 0xb56af65 thru 0xb578fa5 (0xe041 bytes) 2024-09-21T15:34:10.406 INFO:teuthology.orchestra.run.smithi107.stdout:3707 punch from 0xa9bfcb8 to 0xa9cc9bb, (0xcd03 bytes) 2024-09-21T15:34:10.421 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:10.420+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:10.424 INFO:teuthology.orchestra.run.smithi107.stdout:3708 trunc from 0xc2eafad to 0x5b82cc3 2024-09-21T15:34:10.467 INFO:teuthology.orchestra.run.smithi107.stdout:3709 trunc from 0x5b82cc3 to 0xd503b33 2024-09-21T15:34:10.478 INFO:teuthology.orchestra.run.smithi107.stdout:3710 read 0xc19f421 thru 0xc1a9d3c (0xa91c bytes) 2024-09-21T15:34:10.478 INFO:teuthology.orchestra.run.smithi107.stdout:3713 read 0xcc90ab5 thru 0xcc93e64 (0x33b0 bytes) 2024-09-21T15:34:10.478 INFO:teuthology.orchestra.run.smithi107.stdout:3714 write 0x9818b83 thru 0x981f173 (0x65f1 bytes) 2024-09-21T15:34:10.484 INFO:teuthology.orchestra.run.smithi107.stdout:3715 read 0x7cb7183 thru 0x7cb7cdb (0xb59 bytes) 2024-09-21T15:34:10.485 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:10.484+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:10.496 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:10.496 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.505 INFO:teuthology.orchestra.run.smithi107.stdout:3716 trunc from 0xd503b33 to 0x552f19d 2024-09-21T15:34:10.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:10.532 INFO:teuthology.orchestra.run.smithi107.stdout:3717 trunc from 0x552f19d to 0x71a7cde 2024-09-21T15:34:10.542 INFO:teuthology.orchestra.run.smithi107.stdout:3718 write 0x3ba71da thru 0x3bb61b4 (0xefdb bytes) 2024-09-21T15:34:10.548 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.548 INFO:teuthology.orchestra.run.smithi107.stdout:3720 punch from 0x4e1602b to 0x4e1b75e, (0x5733 bytes) 2024-09-21T15:34:10.548 INFO:teuthology.orchestra.run.smithi107.stdout:3721 read 0x869c92 thru 0x873684 (0x99f3 bytes) 2024-09-21T15:34:10.549 INFO:teuthology.orchestra.run.smithi107.stdout:3726 read 0x10b1407 thru 0x10baea7 (0x9aa1 bytes) 2024-09-21T15:34:10.571 INFO:teuthology.orchestra.run.smithi107.stdout:3727 write 0x97f73e8 thru 0x97ffff5 (0x8c0e bytes) 2024-09-21T15:34:10.579 INFO:teuthology.orchestra.run.smithi107.stdout:3729 writesame 0x6b46252 thru 0x6b50e91 data_size 0x20(0xac40 bytes) 2024-09-21T15:34:10.586 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.586 INFO:teuthology.orchestra.run.smithi107.stdout:3731 write 0x3f5fb2a thru 0x3f68d26 (0x91fd bytes) 2024-09-21T15:34:10.599 INFO:teuthology.orchestra.run.smithi107.stdout:3732 trunc from 0x97ffff6 to 0xd5e5ee5 2024-09-21T15:34:10.601 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:10.600+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:10.611 INFO:teuthology.orchestra.run.smithi107.stdout:3733 write 0x6838d1c thru 0x683d2d8 (0x45bd bytes) 2024-09-21T15:34:10.628 INFO:teuthology.orchestra.run.smithi107.stdout:3734 write 0xdc9a44e thru 0xdc9df2c (0x3adf bytes) 2024-09-21T15:34:10.633 INFO:teuthology.orchestra.run.smithi107.stdout:3735 read 0x7838cb2 thru 0x7838cf1 (0x40 bytes) 2024-09-21T15:34:10.633 INFO:teuthology.orchestra.run.smithi107.stdout:3737 read 0x427746c thru 0x427985b (0x23f0 bytes) 2024-09-21T15:34:10.634 INFO:teuthology.orchestra.run.smithi107.stdout:3738 read 0xcdfa65f thru 0xce065d5 (0xbf77 bytes) 2024-09-21T15:34:10.634 INFO:teuthology.orchestra.run.smithi107.stdout:3739 punch from 0xc7b4441 to 0xc7b8ea2, (0x4a61 bytes) 2024-09-21T15:34:10.634 INFO:teuthology.orchestra.run.smithi107.stdout:3740 punch from 0x7cbe225 to 0x7ccb413, (0xd1ee bytes) 2024-09-21T15:34:10.634 INFO:teuthology.orchestra.run.smithi107.stdout:3741 write 0x2e6ff99 thru 0x2e7b64e (0xb6b6 bytes) 2024-09-21T15:34:10.648 INFO:teuthology.orchestra.run.smithi107.stdout:3743 write 0xddbf152 thru 0xddc5873 (0x6722 bytes) 2024-09-21T15:34:10.654 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.654 INFO:teuthology.orchestra.run.smithi107.stdout:3746 read 0xbd3c3e4 thru 0xbd482fe (0xbf1b bytes) 2024-09-21T15:34:10.654 INFO:teuthology.orchestra.run.smithi107.stdout:3748 read 0xb667d9b thru 0xb671a1d (0x9c83 bytes) 2024-09-21T15:34:10.654 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.654 INFO:teuthology.orchestra.run.smithi107.stdout:3750 write 0x115a405 thru 0x116890c (0xe508 bytes) 2024-09-21T15:34:10.657 INFO:teuthology.orchestra.run.smithi107.stdout:3753 write 0x253f772 thru 0x2542c52 (0x34e1 bytes) 2024-09-21T15:34:10.661 INFO:teuthology.orchestra.run.smithi107.stdout:3755 writesame 0xc22af1d thru 0xc231aac data_size 0x8(0x6b90 bytes) 2024-09-21T15:34:10.669 INFO:teuthology.orchestra.run.smithi107.stdout:3756 writesame 0x55db019 thru 0x55e1995 data_size 0xb(0x697d bytes) 2024-09-21T15:34:10.674 INFO:teuthology.orchestra.run.smithi107.stdout:3757 read 0x1ff270a thru 0x1ff7b1a (0x5411 bytes) 2024-09-21T15:34:10.679 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:10.679 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.685 INFO:teuthology.orchestra.run.smithi107.stdout:3758 write 0xe734e72 thru 0xe74439a (0xf529 bytes) 2024-09-21T15:34:10.686 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:10.684+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:10.695 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:10.695 INFO:teuthology.orchestra.run.smithi107.stdout:3759 read 0x6a9b69a thru 0x6aa5c70 (0xa5d7 bytes) 2024-09-21T15:34:10.695 INFO:teuthology.orchestra.run.smithi107.stdout:3760 read 0xd61cb94 thru 0xd626d97 (0xa204 bytes) 2024-09-21T15:34:10.709 INFO:teuthology.orchestra.run.smithi107.stdout:3763 trunc from 0xe74439b to 0x94bec39 2024-09-21T15:34:10.731 INFO:teuthology.orchestra.run.smithi107.stdout:3764 read 0x49e94ff thru 0x49f278d (0x928f bytes) 2024-09-21T15:34:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:3765 write 0x23ca935 thru 0x23d81d3 (0xd89f bytes) 2024-09-21T15:34:10.736 INFO:teuthology.orchestra.run.smithi107.stdout:3767 write 0x509ad06 thru 0x50a7979 (0xcc74 bytes) 2024-09-21T15:34:10.738 INFO:teuthology.orchestra.run.smithi107.stdout:3769 read 0x5f67cc0 thru 0x5f71235 (0x9576 bytes) 2024-09-21T15:34:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:3771 read 0x4d2609 thru 0x4da7ce (0x81c6 bytes) 2024-09-21T15:34:10.756 INFO:teuthology.orchestra.run.smithi107.stdout:3772 write 0xbae8127 thru 0xbaef5f2 (0x74cc bytes) 2024-09-21T15:34:10.764 INFO:teuthology.orchestra.run.smithi107.stdout:3773 write 0xaaefdd9 thru 0xaafe603 (0xe82b bytes) 2024-09-21T15:34:10.769 INFO:teuthology.orchestra.run.smithi107.stdout:3774 read 0x18ae916 thru 0x18aea55 (0x140 bytes) 2024-09-21T15:34:10.770 INFO:teuthology.orchestra.run.smithi107.stdout:3775 read 0x3228921 thru 0x32310c0 (0x87a0 bytes) 2024-09-21T15:34:10.774 INFO:teuthology.orchestra.run.smithi107.stdout:3776 read 0x28b4d17 thru 0x28bab2d (0x5e17 bytes) 2024-09-21T15:34:10.788 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:10.783+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:10.795 INFO:teuthology.orchestra.run.smithi107.stdout:3777 trunc from 0xbaef5f3 to 0x2380746 2024-09-21T15:34:10.838 INFO:teuthology.orchestra.run.smithi107.stdout:3778 write 0x54b615b thru 0x54ba5da (0x4480 bytes) 2024-09-21T15:34:10.844 INFO:teuthology.orchestra.run.smithi107.stdout:3779 read 0x21122c9 thru 0x211df55 (0xbc8d bytes) 2024-09-21T15:34:10.845 INFO:teuthology.orchestra.run.smithi107.stdout:3780 punch from 0x36ae473 to 0x36b4ce5, (0x6872 bytes) 2024-09-21T15:34:10.860 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:10.860 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:10.861 INFO:teuthology.orchestra.run.smithi107.stdout:3783 trunc from 0x54ba5db to 0xb225676 2024-09-21T15:34:10.875 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:10.875 INFO:teuthology.orchestra.run.smithi107.stdout:3784 read 0xc09cf8 thru 0xc10322 (0x662b bytes) 2024-09-21T15:34:10.876 INFO:teuthology.orchestra.run.smithi107.stdout:3785 writesame 0x246308d thru 0x24635d4 data_size 0x1(0x548 bytes) 2024-09-21T15:34:10.880 INFO:teuthology.orchestra.run.smithi107.stdout:3786 writesame 0x54f1825 thru 0x54fa6b2 data_size 0x2(0x8e8e bytes) 2024-09-21T15:34:10.888 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:10.887+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:3787 read 0x2759cb0 thru 0x2761492 (0x77e3 bytes) 2024-09-21T15:34:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:3789 read 0xb22564d thru 0xb225675 (0x29 bytes) 2024-09-21T15:34:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:3790 punch from 0xa19ea01 to 0xa1a2754, (0x3d53 bytes) 2024-09-21T15:34:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:3791 punch from 0x71bb672 to 0x71bed43, (0x36d1 bytes) 2024-09-21T15:34:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:3793 write 0x49f8ffc thru 0x4a07ec7 (0xeecc bytes) 2024-09-21T15:34:10.919 INFO:teuthology.orchestra.run.smithi107.stdout:3794 read 0x1e8d2ca thru 0x1e949d2 (0x7709 bytes) 2024-09-21T15:34:10.920 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.920 INFO:teuthology.orchestra.run.smithi107.stdout:3796 read 0x49941fb thru 0x4997e0a (0x3c10 bytes) 2024-09-21T15:34:10.920 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.920 INFO:teuthology.orchestra.run.smithi107.stdout:3798 write 0x631fb3d thru 0x6321a75 (0x1f39 bytes) 2024-09-21T15:34:10.926 INFO:teuthology.orchestra.run.smithi107.stdout:3799 punch from 0x6012d3d to 0x6019ee4, (0x71a7 bytes) 2024-09-21T15:34:10.927 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.927 INFO:teuthology.orchestra.run.smithi107.stdout:3802 read 0x20a63cc thru 0x20afcdc (0x9911 bytes) 2024-09-21T15:34:10.927 INFO:teuthology.orchestra.run.smithi107.stdout:3803 write 0x6e8b6ec thru 0x6e91160 (0x5a75 bytes) 2024-09-21T15:34:10.933 INFO:teuthology.orchestra.run.smithi107.stdout:3804 read 0x931eba7 thru 0x931f565 (0x9bf bytes) 2024-09-21T15:34:10.933 INFO:teuthology.orchestra.run.smithi107.stdout:3805 read 0x55486e4 thru 0x554e72b (0x6048 bytes) 2024-09-21T15:34:10.933 INFO:teuthology.orchestra.run.smithi107.stdout:3807 writesame 0x979e649 thru 0x97ac2c2 data_size 0x17(0xdc7a bytes) 2024-09-21T15:34:10.949 INFO:teuthology.orchestra.run.smithi107.stdout:3808 trunc from 0xb225676 to 0x86115c6 2024-09-21T15:34:10.963 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.963 INFO:teuthology.orchestra.run.smithi107.stdout:3810 punch from 0x59f2d76 to 0x59f7165, (0x43ef bytes) 2024-09-21T15:34:10.963 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.963 INFO:teuthology.orchestra.run.smithi107.stdout:3815 read 0x8609826 thru 0x86115c5 (0x7da0 bytes) 2024-09-21T15:34:10.963 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:10.964 INFO:teuthology.orchestra.run.smithi107.stdout:3817 write 0x1fc5b92 thru 0x1fd19dc (0xbe4b bytes) 2024-09-21T15:34:10.967 INFO:teuthology.orchestra.run.smithi107.stdout:3818 write 0x1f92c0 thru 0x1fe3bd (0x50fe bytes) 2024-09-21T15:34:10.989 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:10.987+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:10.992 INFO:teuthology.orchestra.run.smithi107.stdout:3819 writesame 0xa4b97c0 thru 0xa4c8127 data_size 0x2(0xe968 bytes) 2024-09-21T15:34:10.992 INFO:teuthology.orchestra.run.smithi107.stdout:3821 write 0x9f9af63 thru 0x9faab97 (0xfc35 bytes) 2024-09-21T15:34:10.999 INFO:teuthology.orchestra.run.smithi107.stdout:3822 read 0x884d76b thru 0x885a142 (0xc9d8 bytes) 2024-09-21T15:34:10.999 INFO:teuthology.orchestra.run.smithi107.stdout:3823 read 0x7caf427 thru 0x7cb4cfa (0x58d4 bytes) 2024-09-21T15:34:11.000 INFO:teuthology.orchestra.run.smithi107.stdout:3824 punch from 0x6e4d670 to 0x6e50c55, (0x35e5 bytes) 2024-09-21T15:34:11.000 INFO:teuthology.orchestra.run.smithi107.stdout:3825 write 0x5116a59 thru 0x511bff9 (0x55a1 bytes) 2024-09-21T15:34:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:11.003 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:11.003 INFO:teuthology.orchestra.run.smithi107.stdout:3826 write 0x8a2eb63 thru 0x8a2eda4 (0x242 bytes) 2024-09-21T15:34:11.010 INFO:teuthology.orchestra.run.smithi107.stdout:3827 read 0x19860b3 thru 0x19944c0 (0xe40e bytes) 2024-09-21T15:34:11.011 INFO:teuthology.orchestra.run.smithi107.stdout:3830 read 0x866a4d3 thru 0x8675768 (0xb296 bytes) 2024-09-21T15:34:11.012 INFO:teuthology.orchestra.run.smithi107.stdout:3831 read 0xa92993 thru 0xa9c841 (0x9eaf bytes) 2024-09-21T15:34:11.017 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:11.033 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:11.033 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:11.033 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T15:34:11.033 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T15:34:11.033 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T15:34:11.034 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-21T15:34:11.038 INFO:teuthology.orchestra.run.smithi107.stdout:3832 trunc from 0xa4c8128 to 0x77690 2024-09-21T15:34:11.072 INFO:teuthology.orchestra.run.smithi107.stdout:3836 punch from 0x75f12 to 0x77690, (0x177e bytes) 2024-09-21T15:34:11.073 INFO:teuthology.orchestra.run.smithi107.stdout:3837 punch from 0x10b4e to 0x1966b, (0x8b1d bytes) 2024-09-21T15:34:11.073 INFO:teuthology.orchestra.run.smithi107.stdout:3838 punch from 0x70cd1 to 0x77690, (0x69bf bytes) 2024-09-21T15:34:11.090 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:11.088+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:11.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:11.105 INFO:teuthology.orchestra.run.smithi107.stdout:3840 write 0x9f61ca9 thru 0x9f630b0 (0x1408 bytes) 2024-09-21T15:34:11.109 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.124 INFO:teuthology.orchestra.run.smithi107.stdout:3842 write 0xe30b151 thru 0xe30c119 (0xfc9 bytes) 2024-09-21T15:34:11.129 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.129 INFO:teuthology.orchestra.run.smithi107.stdout:3844 write 0x9c5dcb4 thru 0x9c5de15 (0x162 bytes) 2024-09-21T15:34:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.133 INFO:teuthology.orchestra.run.smithi107.stdout:3848 write 0x15fd11a thru 0x15ff29d (0x2184 bytes) 2024-09-21T15:34:11.138 INFO:teuthology.orchestra.run.smithi107.stdout:3849 write 0x1a6f783 thru 0x1a7c8e7 (0xd165 bytes) 2024-09-21T15:34:11.140 INFO:teuthology.orchestra.run.smithi107.stdout:3851 read 0x69d3646 thru 0x69dcf30 (0x98eb bytes) 2024-09-21T15:34:11.140 INFO:teuthology.orchestra.run.smithi107.stdout:3852 write 0x595bf1c thru 0x5969db3 (0xde98 bytes) 2024-09-21T15:34:11.151 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:11.152 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:11.157 INFO:teuthology.orchestra.run.smithi107.stdout:3853 trunc from 0xe30c11a to 0x6df52b8 2024-09-21T15:34:11.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:11.174 INFO:teuthology.orchestra.run.smithi107.stdout:3855 punch from 0x2e46274 to 0x2e4cbaa, (0x6936 bytes) 2024-09-21T15:34:11.174 INFO:teuthology.orchestra.run.smithi107.stdout:3856 punch from 0x1f8ceb3 to 0x1f8e9c0, (0x1b0d bytes) 2024-09-21T15:34:11.177 INFO:teuthology.orchestra.run.smithi107.stdout:3857 trunc from 0x6df52b8 to 0x50bb3ee 2024-09-21T15:34:11.182 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:11.183 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T15:34:11.183 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:11.189 INFO:teuthology.orchestra.run.smithi107.stdout:3859 read 0x25281eb thru 0x25293af (0x11c5 bytes) 2024-09-21T15:34:11.189 INFO:teuthology.orchestra.run.smithi107.stdout:3860 read 0xd2877d thru 0xd310be (0x8942 bytes) 2024-09-21T15:34:11.209 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:11.208+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:11.210 INFO:teuthology.orchestra.run.smithi107.stdout:3862 writesame 0xb85565f thru 0xb858b53 data_size 0x1(0x34f5 bytes) 2024-09-21T15:34:11.219 INFO:teuthology.orchestra.run.smithi107.stdout:3863 trunc from 0xb858b54 to 0x54009c7 2024-09-21T15:34:11.246 INFO:teuthology.orchestra.run.smithi107.stdout:3864 writesame 0xc6ebdf5 thru 0xc6f718d data_size 0x1(0xb399 bytes) 2024-09-21T15:34:11.246 INFO:teuthology.orchestra.run.smithi107.stdout:3865 write 0x8928daf thru 0x892e78b (0x59dd bytes) 2024-09-21T15:34:11.251 INFO:teuthology.orchestra.run.smithi107.stdout:3866 trunc from 0xc6f718e to 0xd22df34 2024-09-21T15:34:11.253 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:11.253 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:11.277 INFO:teuthology.orchestra.run.smithi107.stdout:3867 trunc from 0xd22df34 to 0xe5b434 2024-09-21T15:34:11.282 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:11.283 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:11.290 INFO:teuthology.orchestra.run.smithi107.stdout:3868 trunc from 0xe5b434 to 0x1540d46 2024-09-21T15:34:11.291 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:11.287+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:11.296 INFO:teuthology.orchestra.run.smithi107.stdout:3869 punch from 0xbffb6e to 0xc09e84, (0xa316 bytes) 2024-09-21T15:34:11.303 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:11.303 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:11.311 INFO:teuthology.orchestra.run.smithi107.stdout:3872 write 0x760d03d thru 0x760f9ef (0x29b3 bytes) 2024-09-21T15:34:11.316 INFO:teuthology.orchestra.run.smithi107.stdout:3873 read 0x39196db thru 0x39278f6 (0xe21c bytes) 2024-09-21T15:34:11.316 INFO:teuthology.orchestra.run.smithi107.stdout:3874 write 0x322300c thru 0x32234e9 (0x4de bytes) 2024-09-21T15:34:11.320 INFO:teuthology.orchestra.run.smithi107.stdout:3875 read 0x5ac189 thru 0x5ae2ae (0x2126 bytes) 2024-09-21T15:34:11.322 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:11.322 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:11.334 INFO:teuthology.orchestra.run.smithi107.stdout:3876 write 0xb3bface thru 0xb3c14ca (0x19fd bytes) 2024-09-21T15:34:11.340 INFO:teuthology.orchestra.run.smithi107.stdout:3877 write 0x8a45c7a thru 0x8a52720 (0xcaa7 bytes) 2024-09-21T15:34:11.341 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:11.341 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:11.342 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:11.342 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:11.348 INFO:teuthology.orchestra.run.smithi107.stdout:3878 trunc from 0xb3c14cb to 0x819e836 2024-09-21T15:34:11.358 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:11.359 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:11.359 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:11.364 INFO:teuthology.orchestra.run.smithi107.stdout:3879 read 0x3974bbb thru 0x397c24e (0x7694 bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3880 read 0x62b87e8 thru 0x62c1c14 (0x942d bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3881 punch from 0x6dad3cf to 0x6daec32, (0x1863 bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3884 punch from 0x44044e9 to 0x44049b1, (0x4c8 bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3885 read 0x2d21be1 thru 0x2d2dbe4 (0xc004 bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3886 read 0x3288745 thru 0x32917fd (0x90b9 bytes) 2024-09-21T15:34:11.365 INFO:teuthology.orchestra.run.smithi107.stdout:3887 write 0x59d863d thru 0x59d9a65 (0x1429 bytes) 2024-09-21T15:34:11.368 INFO:teuthology.orchestra.run.smithi107.stdout:3888 write 0x1e18cce thru 0x1e19d68 (0x109b bytes) 2024-09-21T15:34:11.377 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:11.377 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:11.381 INFO:teuthology.orchestra.run.smithi107.stdout:3889 writesame 0xaaa9850 thru 0xaab8523 data_size 0x2e(0xecd4 bytes) 2024-09-21T15:34:11.381 INFO:teuthology.orchestra.run.smithi107.stdout:3890 write 0x9d165c4 thru 0x9d2459c (0xdfd9 bytes) 2024-09-21T15:34:11.383 INFO:teuthology.orchestra.run.smithi107.stdout:3891 read 0x34ef16f thru 0x34fd63c (0xe4ce bytes) 2024-09-21T15:34:11.383 INFO:teuthology.orchestra.run.smithi107.stdout:3892 writesame 0x6a2acd8 thru 0x6a30313 data_size 0x2(0x563c bytes) 2024-09-21T15:34:11.390 INFO:teuthology.orchestra.run.smithi107.stdout:3893 write 0x2e03bb1 thru 0x2e0d1c3 (0x9613 bytes) 2024-09-21T15:34:11.391 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:11.387+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:11.394 INFO:teuthology.orchestra.run.smithi107.stdout:3895 read 0xa04db7e thru 0xa0534d0 (0x5953 bytes) 2024-09-21T15:34:11.394 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.394 INFO:teuthology.orchestra.run.smithi107.stdout:3897 punch from 0x50e3a2d to 0x50f1dc9, (0xe39c bytes) 2024-09-21T15:34:11.394 INFO:teuthology.orchestra.run.smithi107.stdout:3898 punch from 0x1d4a881 to 0x1d4c2c7, (0x1a46 bytes) 2024-09-21T15:34:11.397 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:11.397 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:11.399 INFO:teuthology.orchestra.run.smithi107.stdout:3899 trunc from 0xaab8524 to 0x6daaebd 2024-09-21T15:34:11.427 INFO:teuthology.orchestra.run.smithi107.stdout:3900 write 0xcbc8c54 thru 0xcbd568b (0xca38 bytes) 2024-09-21T15:34:11.430 INFO:teuthology.orchestra.run.smithi107.stdout:3901 write 0xb46ea39 thru 0xb477099 (0x8661 bytes) 2024-09-21T15:34:11.450 INFO:teuthology.orchestra.run.smithi107.stdout:3902 trunc from 0xcbd568c to 0x19ab481 2024-09-21T15:34:11.488 INFO:teuthology.orchestra.run.smithi107.stdout:3903 writesame 0xb38299a thru 0xb38ac69 data_size 0x70(0x82d0 bytes) 2024-09-21T15:34:11.490 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.712160031, 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.711782039999999, 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.711562785000002, 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.711165621999999, 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.710761833999999, 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.710306072000002, 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.703079526, 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.702713238000001, 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.702001395, 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.698740848, 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.698300628999998, 2024-09-21T15:34:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.697989205999999, 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.673061598, 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.672431941999999, 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.672027479, 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.671489086000001, 2024-09-21T15:34:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.670962068000001, 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.670540342999999, 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.564 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.670124941000001, 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 16.669574514000001, 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:11.578 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:11.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3904 write 0xe7ebef3 thru 0xe7faf4e (0xf05c bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3906 writesame 0x65415f9 thru 0x654c6f0 data_size 0x1c(0xb0f8 bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3907 read 0x4cd2e67 thru 0x4cdc727 (0x98c1 bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3908 write 0x118419d thru 0x11879a1 (0x3805 bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3909 punch from 0x425f11 to 0x42bf66, (0x6055 bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3910 read 0x4db99f1 thru 0x4dc2f8e (0x959e bytes) 2024-09-21T15:34:11.581 INFO:teuthology.orchestra.run.smithi107.stdout:3911 writesame 0x1ca5f4a thru 0x1caf8d7 data_size 0x5(0x998e bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3912 write 0xa9cf410 thru 0xa9d716a (0x7d5b bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3913 writesame 0xb802919 thru 0xb8086f3 data_size 0x1(0x5ddb bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3914 read 0x2fe338f thru 0x2ff20b0 (0xed22 bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3917 read 0x4021612 thru 0x402b955 (0xa344 bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3918 read 0x10aeb39 thru 0x10b71e5 (0x86ad bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3920 read 0x3470bfc thru 0x347a119 (0x951e bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3921 read 0xe42b1c1 thru 0xe43b04c (0xfe8c bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3922 punch from 0x96a2930 to 0x96a60eb, (0x37bb bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3924 read 0x413079e thru 0x413b13e (0xa9a1 bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3926 read 0x5ac344c thru 0x5ac95aa (0x615f bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3927 read 0x67decab thru 0x67e56bb (0x6a11 bytes) 2024-09-21T15:34:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:3928 write 0x3203e8a thru 0x3209419 (0x5590 bytes) 2024-09-21T15:34:11.583 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:11.575+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:11.583 INFO:teuthology.orchestra.run.smithi107.stdout:3932 read 0x7a94b59 thru 0x7aa3194 (0xe63c bytes) 2024-09-21T15:34:11.583 INFO:teuthology.orchestra.run.smithi107.stdout:3933 read 0xaede1ea thru 0xaee0fa2 (0x2db9 bytes) 2024-09-21T15:34:11.585 INFO:teuthology.orchestra.run.smithi107.stdout:3935 read 0x407896b thru 0x407a837 (0x1ecd bytes) 2024-09-21T15:34:11.585 INFO:teuthology.orchestra.run.smithi107.stdout:3937 read 0xc886976 thru 0xc894fe4 (0xe66f bytes) 2024-09-21T15:34:11.585 INFO:teuthology.orchestra.run.smithi107.stdout:3939 read 0xe342a83 thru 0xe343a98 (0x1016 bytes) 2024-09-21T15:34:11.585 INFO:teuthology.orchestra.run.smithi107.stdout:3941 writesame 0x22c15b0 thru 0x22cddd5 data_size 0x89(0xc826 bytes) 2024-09-21T15:34:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:3943 read 0x899dd19 thru 0x89a3352 (0x563a bytes) 2024-09-21T15:34:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:3944 punch from 0xe625701 to 0xe6287ae, (0x30ad bytes) 2024-09-21T15:34:11.589 INFO:teuthology.orchestra.run.smithi107.stdout:3945 write 0x1adabc thru 0x1aea3a (0xf7f bytes) 2024-09-21T15:34:11.592 INFO:teuthology.orchestra.run.smithi107.stdout:3946 writesame 0x55690aa thru 0x557488b data_size 0x1(0xb7e2 bytes) 2024-09-21T15:34:11.659 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.659 INFO:teuthology.orchestra.run.smithi107.stdout:3948 writesame 0xb40ad09 thru 0xb40fc8b data_size 0x17(0x4f83 bytes) 2024-09-21T15:34:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:3949 read 0xd6bfd7b thru 0xd6c3908 (0x3b8e bytes) 2024-09-21T15:34:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:3954 punch from 0x89054c9 to 0x8908cc6, (0x37fd bytes) 2024-09-21T15:34:11.667 INFO:teuthology.orchestra.run.smithi107.stdout:3956 writesame 0x7249040 thru 0x724b828 data_size 0x1(0x27e9 bytes) 2024-09-21T15:34:11.680 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:11.679+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:11.681 INFO:teuthology.orchestra.run.smithi107.stdout:3958 write 0x4f9e327 thru 0x4fa1316 (0x2ff0 bytes) 2024-09-21T15:34:11.695 INFO:teuthology.orchestra.run.smithi107.stdout:3959 trunc from 0xe7faf4f to 0x9ed4b29 2024-09-21T15:34:11.713 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:11.713 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:11.713 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:11.717 INFO:teuthology.orchestra.run.smithi107.stdout:3960 read 0x5601582 thru 0x5603012 (0x1a91 bytes) 2024-09-21T15:34:11.717 INFO:teuthology.orchestra.run.smithi107.stdout:3961 writesame 0x2dafde8 thru 0x2db7a2b data_size 0x2c(0x7c44 bytes) 2024-09-21T15:34:11.724 INFO:teuthology.orchestra.run.smithi107.stdout:3964 read 0x2585316 thru 0x259130e (0xbff9 bytes) 2024-09-21T15:34:11.725 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.725 INFO:teuthology.orchestra.run.smithi107.stdout:3966 write 0x50f8cc5 thru 0x50fc634 (0x3970 bytes) 2024-09-21T15:34:11.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:3967 punch from 0x2ad2f7d to 0x2ad7e45, (0x4ec8 bytes) 2024-09-21T15:34:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:3968 punch from 0x6d549cc to 0x6d59835, (0x4e69 bytes) 2024-09-21T15:34:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:3969 punch from 0xd771b0 to 0xd7db73, (0x69c3 bytes) 2024-09-21T15:34:11.731 INFO:teuthology.orchestra.run.smithi107.stdout:3970 write 0x6a85062 thru 0x6a9392e (0xe8cd bytes) 2024-09-21T15:34:11.734 INFO:teuthology.orchestra.run.smithi107.stdout:3972 write 0x1f83488 thru 0x1f8dbb3 (0xa72c bytes) 2024-09-21T15:34:11.742 INFO:teuthology.orchestra.run.smithi107.stdout:3974 trunc from 0x9ed4b29 to 0xd0af7a4 2024-09-21T15:34:11.753 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.753 INFO:teuthology.orchestra.run.smithi107.stdout:3978 read 0x6e0e484 thru 0x6e1ca54 (0xe5d1 bytes) 2024-09-21T15:34:11.754 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.754 INFO:teuthology.orchestra.run.smithi107.stdout:3981 write 0x7d73f76 thru 0x7d7a57c (0x6607 bytes) 2024-09-21T15:34:11.761 INFO:teuthology.orchestra.run.smithi107.stdout:3982 read 0x7f18f52 thru 0x7f267b0 (0xd85f bytes) 2024-09-21T15:34:11.761 INFO:teuthology.orchestra.run.smithi107.stdout:3983 read 0x2b4ed81 thru 0x2b5b87f (0xcaff bytes) 2024-09-21T15:34:11.762 INFO:teuthology.orchestra.run.smithi107.stdout:3984 writesame 0xf4ddac thru 0xf57bda data_size 0x7(0x9e2f bytes) 2024-09-21T15:34:11.771 INFO:teuthology.orchestra.run.smithi107.stdout:3985 writesame 0xc9e3d01 thru 0xc9f0a09 data_size 0x1(0xcd09 bytes) 2024-09-21T15:34:11.781 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:11.780+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:11.843 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.843 INFO:teuthology.orchestra.run.smithi107.stdout:3987 write 0xc2d9e54 thru 0xc2e6ddd (0xcf8a bytes) 2024-09-21T15:34:11.848 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:11.848 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:11.849 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:11.849 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:11.849 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:11.851 INFO:teuthology.orchestra.run.smithi107.stdout:3989 read 0x82b2451 thru 0x82c018b (0xdd3b bytes) 2024-09-21T15:34:11.851 INFO:teuthology.orchestra.run.smithi107.stdout:3990 punch from 0x65a3bed to 0x65a4028, (0x43b bytes) 2024-09-21T15:34:11.851 INFO:teuthology.orchestra.run.smithi107.stdout:3993 write 0x56bd204 thru 0x56c2cda (0x5ad7 bytes) 2024-09-21T15:34:11.855 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.855 INFO:teuthology.orchestra.run.smithi107.stdout:3995 write 0xa7314ea thru 0xa73ae38 (0x994f bytes) 2024-09-21T15:34:11.861 INFO:teuthology.orchestra.run.smithi107.stdout:3997 read 0x13dbfe7 thru 0x13e4c73 (0x8c8d bytes) 2024-09-21T15:34:11.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:11.877 INFO:teuthology.orchestra.run.smithi107.stdout:3998 write 0xdfef769 thru 0xdff109f (0x1937 bytes) 2024-09-21T15:34:11.883 INFO:teuthology.orchestra.run.smithi107.stdout:3999 read 0x383e37c thru 0x384a22b (0xbeb0 bytes) 2024-09-21T15:34:11.883 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:11.879+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:11.884 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.884 INFO:teuthology.orchestra.run.smithi107.stdout:4001 read 0x83322d2 thru 0x833579f (0x34ce bytes) 2024-09-21T15:34:11.884 INFO:teuthology.orchestra.run.smithi107.stdout:4002 writesame 0x80a7a8 thru 0x813151 data_size 0x43(0x89aa bytes) 2024-09-21T15:34:11.888 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.888 INFO:teuthology.orchestra.run.smithi107.stdout:4005 write 0x30b5ff8 thru 0x30bc9f3 (0x69fc bytes) 2024-09-21T15:34:11.904 INFO:teuthology.orchestra.run.smithi107.stdout:4007 write 0xe9a8b56 thru 0xe9b453a (0xb9e5 bytes) 2024-09-21T15:34:11.911 INFO:teuthology.orchestra.run.smithi107.stdout:4008 read 0xa9974b8 thru 0xa9a1f66 (0xaaaf bytes) 2024-09-21T15:34:11.911 INFO:teuthology.orchestra.run.smithi107.stdout:4009 write 0x8ccf709 thru 0x8cd57d5 (0x60cd bytes) 2024-09-21T15:34:11.917 INFO:teuthology.orchestra.run.smithi107.stdout:4010 read 0x398f907 thru 0x399057d (0xc77 bytes) 2024-09-21T15:34:11.918 INFO:teuthology.orchestra.run.smithi107.stdout:4011 read 0x57af3f2 thru 0x57bc88d (0xd49c bytes) 2024-09-21T15:34:11.919 INFO:teuthology.orchestra.run.smithi107.stdout:4012 read 0xb3bbbf1 thru 0xb3c1296 (0x56a6 bytes) 2024-09-21T15:34:11.919 INFO:teuthology.orchestra.run.smithi107.stdout:4014 read 0x7e280a3 thru 0x7e2de32 (0x5d90 bytes) 2024-09-21T15:34:11.919 INFO:teuthology.orchestra.run.smithi107.stdout:4016 read 0xb863fbe thru 0xb86ea44 (0xaa87 bytes) 2024-09-21T15:34:11.919 INFO:teuthology.orchestra.run.smithi107.stdout:4017 read 0xc501ef2 thru 0xc511161 (0xf270 bytes) 2024-09-21T15:34:11.920 INFO:teuthology.orchestra.run.smithi107.stdout:4018 write 0xab2a0fb thru 0xab2ddf1 (0x3cf7 bytes) 2024-09-21T15:34:11.923 INFO:teuthology.orchestra.run.smithi107.stdout:4019 read 0xe5719e8 thru 0xe57ad6b (0x9384 bytes) 2024-09-21T15:34:11.924 INFO:teuthology.orchestra.run.smithi107.stdout:4021 punch from 0x74f3a45 to 0x74f73d9, (0x3994 bytes) 2024-09-21T15:34:11.937 INFO:teuthology.orchestra.run.smithi107.stdout:4022 trunc from 0xe9b453b to 0x6081fbc 2024-09-21T15:34:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:4024 read 0x55f4745 thru 0x55fa23f (0x5afb bytes) 2024-09-21T15:34:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:4026 punch from 0x3aea70 to 0x3bc8a6, (0xde36 bytes) 2024-09-21T15:34:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:4027 write 0x289ecd0 thru 0x28a07d6 (0x1b07 bytes) 2024-09-21T15:34:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:4030 writesame 0x34decbc thru 0x34ed138 data_size 0x1(0xe47d bytes) 2024-09-21T15:34:11.984 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:11.980+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:12.002 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.002 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 73.290144897000005, 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:34:12.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:34:12.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:12.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.008 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.352325377, 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.009 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:34:12.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:34:12.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:12.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124484 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2975744~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:12.014 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286958670000002, 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043060238000000001, 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124484 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371116+0000", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371118+0000", 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9460000000000001e-06 2024-09-21T15:34:12.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371123+0000", 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3700000000000003e-06 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371127+0000", 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0280000000000002e-06 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414044+0000", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042916081000000002 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414115+0000", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1607999999999996e-05 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414177+0000", 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1617000000000003e-05 2024-09-21T15:34:12.018 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124485 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2985984~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286813369999997, 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043229165, 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124485 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371133+0000", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8899999999999999e-06 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371138+0000", 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1359999999999996e-06 2024-09-21T15:34:12.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371147+0000", 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8340000000000005e-06 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414210+0000", 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043063203000000001 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414291+0000", 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1428999999999996e-05 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414360+0000", 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9033000000000001e-05 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124486 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2996224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286622369999999, 2024-09-21T15:34:12.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043374530000000001, 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124486 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371152+0000", 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.888e-06 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371157+0000", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2560000000000002e-06 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371161+0000", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0899999999999998e-06 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414392+0000", 2024-09-21T15:34:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043231251999999998 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414461+0000", 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8369999999999998e-05 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414525+0000", 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4046000000000003e-05 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124487 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3006464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286478190000002, 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043541654999999999, 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124487 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371164+0000", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371166+0000", 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7460000000000001e-06 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371171+0000", 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0719999999999999e-06 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371175+0000", 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0659999999999997e-06 2024-09-21T15:34:12.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414556+0000", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043381182999999997 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414636+0000", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9809999999999997e-05 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414706+0000", 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0134000000000003e-05 2024-09-21T15:34:12.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124488 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3016704~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286339549999997, 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692364999999997, 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124488 2024-09-21T15:34:12.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:12.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371180+0000", 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.734e-06 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371185+0000", 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371189+0000", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.912e-06 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414738+0000", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043548877 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414809+0000", 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1675999999999996e-05 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414871+0000", 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1477999999999997e-05 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124489 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3026944~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.628620019, 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043866982999999998, 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124489 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371194+0000", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371199+0000", 2024-09-21T15:34:12.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0220000000000004e-06 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371203+0000", 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.941e-06 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414912+0000", 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043709439000000003 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414992+0000", 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9572000000000003e-05 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415059+0000", 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7539000000000005e-05 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124490 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3037184~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6286061379999999, 2024-09-21T15:34:12.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044018785999999997, 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124490 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371208+0000", 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8080000000000001e-06 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371213+0000", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371217+0000", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0300000000000004e-06 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415092+0000", 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043875441000000001 2024-09-21T15:34:12.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415163+0000", 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1026999999999996e-05 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415225+0000", 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1793999999999998e-05 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124491 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3047424~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6285920740000002, 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044189697999999999, 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124491 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371230+0000", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0068000000000001e-05 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371244+0000", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.395e-05 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371251+0000", 2024-09-21T15:34:12.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1960000000000002e-06 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415258+0000", 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044007019000000001 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415339+0000", 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1036000000000001e-05 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415410+0000", 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0784999999999994e-05 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124492 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3057664~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6285366539999999, 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044296730999999999, 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124492 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371275+0000", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371279+0000", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5260000000000002e-06 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371291+0000", 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2098e-05 2024-09-21T15:34:12.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371307+0000", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6787999999999999e-05 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415440+0000", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044132678000000002 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415510+0000", 2024-09-21T15:34:12.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9999999999999994e-05 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415572+0000", 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2333000000000001e-05 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124495 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3088384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6268463230000001, 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043138292000000002, 2024-09-21T15:34:12.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124495 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372965+0000", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372969+0000", 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0119999999999999e-06 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372982+0000", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2517000000000001e-05 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372991+0000", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4180000000000003e-06 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415952+0000", 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042960920999999999 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416034+0000", 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1464999999999993e-05 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416104+0000", 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0635000000000004e-05 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124496 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3098624~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6268107600000001, 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043270034999999998, 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124496 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373001+0000", 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373005+0000", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7340000000000002e-06 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373020+0000", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5622000000000001e-05 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373029+0000", 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0240000000000003e-06 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416137+0000", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043107686999999999 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416206+0000", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999995e-05 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416272+0000", 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5283999999999994e-05 2024-09-21T15:34:12.061 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124497 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3108864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.62677381, 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043416800999999998, 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124497 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373038+0000", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373041+0000", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5880000000000002e-06 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373052+0000", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.057e-05 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373060+0000", 2024-09-21T15:34:12.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8720000000000002e-06 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416303+0000", 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043243216000000001 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416386+0000", 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2712000000000004e-05 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416455+0000", 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9535000000000004e-05 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124498 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3119104~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6267435040000002, 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043553477, 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.066 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124498 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373068+0000", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373072+0000", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3220000000000001e-06 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373082+0000", 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0368e-05 2024-09-21T15:34:12.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373100+0000", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8309e-05 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416488+0000", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043387362999999998 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416560+0000", 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2568e-05 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416622+0000", 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2212999999999995e-05 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124499 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3129344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6267047149999998, 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692511000000003, 2024-09-21T15:34:12.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124499 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373107+0000", 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373111+0000", 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373122+0000", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1590000000000001e-05 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373131+0000", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3519999999999996e-06 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416653+0000", 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043521988999999997 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416733+0000", 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9479000000000003e-05 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416800+0000", 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7218999999999995e-05 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124500 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3139584~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.074 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.626662965, 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043824758999999998, 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124500 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:12.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373152+0000", 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.58e-06 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373166+0000", 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3835999999999999e-05 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373176+0000", 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5019999999999998e-06 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416833+0000", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043656815000000002 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416911+0000", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8788999999999994e-05 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416974+0000", 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2854999999999994e-05 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.078 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124521 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4077568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6221896440000001, 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043139638000000001, 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124521 2024-09-21T15:34:12.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377622+0000", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377626+0000", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377638+0000", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2216e-05 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377653+0000", 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5114e-05 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420629+0000", 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042976331 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420699+0000", 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9349000000000003e-05 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420762+0000", 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3812000000000005e-05 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124522 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3997696~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6221505380000001, 2024-09-21T15:34:12.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043278491000000002, 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124522 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377661+0000", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377665+0000", 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:12.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377675+0000", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0621999999999999e-05 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377695+0000", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9592000000000001e-05 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420796+0000", 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043101511000000002 2024-09-21T15:34:12.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420877+0000", 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0694999999999999e-05 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420940+0000", 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3170999999999995e-05 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124523 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4087808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 3.6220348389999999, 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043336070999999997, 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124523 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377702+0000", 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999242 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377782+0000", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0505000000000003e-05 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377790+0000", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2740000000000007e-06 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377796+0000", 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420979+0000", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043182812000000001 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421048+0000", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9449999999999994e-05 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421114+0000", 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5278999999999996e-05 2024-09-21T15:34:12.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.092 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.092 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.092 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.092 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.092 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.092 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:34:12.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:12.094 INFO:teuthology.orchestra.run.smithi107.stdout:4031 writesame 0x510d686 thru 0x5111811 data_size 0xa(0x418c bytes) 2024-09-21T15:34:12.094 INFO:teuthology.orchestra.run.smithi107.stdout:4032 read 0xe98439 thru 0xe9957c (0x1144 bytes) 2024-09-21T15:34:12.095 INFO:teuthology.orchestra.run.smithi107.stdout:4033 trunc from 0x6081fbc to 0x11976cd 2024-09-21T15:34:12.095 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:12.092+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:12.113 INFO:teuthology.orchestra.run.smithi107.stdout:4034 writesame 0x41cdec1 thru 0x41d4732 data_size 0x1d(0x6872 bytes) 2024-09-21T15:34:12.116 INFO:teuthology.orchestra.run.smithi107.stdout:4035 trunc from 0x41d4733 to 0x4e818b9 2024-09-21T15:34:12.143 INFO:teuthology.orchestra.run.smithi107.stdout:4036 write 0x72760bc thru 0x727a92d (0x4872 bytes) 2024-09-21T15:34:12.148 INFO:teuthology.orchestra.run.smithi107.stdout:4037 punch from 0x15ff19a to 0x160b3cb, (0xc231 bytes) 2024-09-21T15:34:12.149 INFO:teuthology.orchestra.run.smithi107.stdout:4038 write 0xb31d23 thru 0xb3dbee (0xbecc bytes) 2024-09-21T15:34:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:4040 write 0x54ec777 thru 0x54ed16f (0x9f9 bytes) 2024-09-21T15:34:12.163 INFO:teuthology.orchestra.run.smithi107.stdout:4041 trunc from 0x727a92e to 0xaa2d6f3 2024-09-21T15:34:12.187 INFO:teuthology.orchestra.run.smithi107.stdout:4042 trunc from 0xaa2d6f3 to 0x1e2331b 2024-09-21T15:34:12.194 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:12.191+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:12.211 INFO:teuthology.orchestra.run.smithi107.stdout:4044 read 0x5021a0 thru 0x509e41 (0x7ca2 bytes) 2024-09-21T15:34:12.211 INFO:teuthology.orchestra.run.smithi107.stdout:4046 punch from 0x1d91b67 to 0x1d9f18c, (0xd625 bytes) 2024-09-21T15:34:12.227 INFO:teuthology.orchestra.run.smithi107.stdout:4047 write 0x5b68c5f thru 0x5b753d2 (0xc774 bytes) 2024-09-21T15:34:12.235 INFO:teuthology.orchestra.run.smithi107.stdout:4049 punch from 0x314e716 to 0x314f7b9, (0x10a3 bytes) 2024-09-21T15:34:12.235 INFO:teuthology.orchestra.run.smithi107.stdout:4050 write 0x48489a7 thru 0x4854467 (0xbac1 bytes) 2024-09-21T15:34:12.241 INFO:teuthology.orchestra.run.smithi107.stdout:4052 punch from 0x5b2b2db to 0x5b357c4, (0xa4e9 bytes) 2024-09-21T15:34:12.241 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:12.249 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:12.249 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:12.249 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.262 INFO:teuthology.orchestra.run.smithi107.stdout:4054 write 0xc2a25f2 thru 0xc2a589b (0x32aa bytes) 2024-09-21T15:34:12.266 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:12.267 INFO:teuthology.orchestra.run.smithi107.stdout:4055 read 0x6d803f thru 0x6dedf3 (0x6db5 bytes) 2024-09-21T15:34:12.269 INFO:teuthology.orchestra.run.smithi107.stdout:4057 write 0x9059f22 thru 0x9064f3f (0xb01e bytes) 2024-09-21T15:34:12.273 INFO:teuthology.orchestra.run.smithi107.stdout:4062 punch from 0x527c6b5 to 0x52848e4, (0x822f bytes) 2024-09-21T15:34:12.273 INFO:teuthology.orchestra.run.smithi107.stdout:4064 write 0x99e8d67 thru 0x99eeab3 (0x5d4d bytes) 2024-09-21T15:34:12.277 INFO:teuthology.orchestra.run.smithi107.stdout:4065 read 0x1a93071 thru 0x1a9909b (0x602b bytes) 2024-09-21T15:34:12.278 INFO:teuthology.orchestra.run.smithi107.stdout:4067 write 0x9d2a009 thru 0x9d2cbef (0x2be7 bytes) 2024-09-21T15:34:12.285 INFO:teuthology.orchestra.run.smithi107.stdout:4068 writesame 0x764ae6e thru 0x76552fa data_size 0x19(0xa48d bytes) 2024-09-21T15:34:12.293 INFO:teuthology.orchestra.run.smithi107.stdout:4069 read 0xa31e76a thru 0xa328276 (0x9b0d bytes) 2024-09-21T15:34:12.293 INFO:teuthology.orchestra.run.smithi107.stdout:4070 punch from 0x2943117 to 0x294f268, (0xc151 bytes) 2024-09-21T15:34:12.295 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:12.292+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:12.305 INFO:teuthology.orchestra.run.smithi107.stdout:4071 write 0xd56f9d7 thru 0xd5730b7 (0x36e1 bytes) 2024-09-21T15:34:12.311 INFO:teuthology.orchestra.run.smithi107.stdout:4072 punch from 0xc7428df to 0xc7498df, (0x7000 bytes) 2024-09-21T15:34:12.311 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:12.311 INFO:teuthology.orchestra.run.smithi107.stdout:4075 writesame 0x53b82e2 thru 0x53c620c data_size 0x1(0xdf2b bytes) 2024-09-21T15:34:12.349 INFO:teuthology.orchestra.run.smithi107.stdout:4076 punch from 0xc8301cb to 0xc8350d7, (0x4f0c bytes) 2024-09-21T15:34:12.354 INFO:teuthology.orchestra.run.smithi107.stdout:4077 trunc from 0xd5730b8 to 0xa360993 2024-09-21T15:34:12.381 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:12.381 INFO:teuthology.orchestra.run.smithi107.stdout:4082 clone 29 order 23 su 1048576 sc 15 2024-09-21T15:34:12.392 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.392 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:12.392 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:12.392 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:12.392 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.396 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:12.392+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:12.406 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:12.497 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:12.496+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:12.568 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.568 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:12.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:12.568 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:12.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 57.442729368000002, 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:12.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:12.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 33.000573154999998, 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:12.580 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 33.000548408999997, 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:12.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 33.000503459000001, 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:12.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996646800999997, 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:12.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996602441, 2024-09-21T15:34:12.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:12.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:12.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996549092000002, 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.598 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.602 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996517699000002, 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:12.606 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996471972999998, 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.607 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996442600000002, 2024-09-21T15:34:12.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:12.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.615 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996418040000002, 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:12.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.996391402999997, 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.622 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992552836000002, 2024-09-21T15:34:12.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:12.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992523016, 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:12.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992497485999998, 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.633 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:12.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:12.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:12.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992471780000002, 2024-09-21T15:34:12.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:12.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.641 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992421739999997, 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.642 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992386940000003, 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.646 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:12.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:12.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.992362290999999, 2024-09-21T15:34:12.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:12.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.654 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.988311883999998, 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.655 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:12.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:12.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:12.659 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:12.661 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:12.659+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:12.761 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:12.760+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:12.830 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:12.830 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:12.892 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:12.891+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:12.963 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:12.959+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:12.964 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:12.964 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:12.964 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:12.964 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:12.964 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:12.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:13.052 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:34:13.063 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:13.059+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:13.150 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:13.150 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.604462230999999, 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:34:13.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.133545960000006, 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:34:13.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:34:13.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.159 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.123755868000003, 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:13.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.163 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.109801828000002, 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:34:13.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.101634683, 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.172 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.100888284000007, 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:13.173 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.174 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.082497308000001, 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.177 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:34:13.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:34:13.180 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.021631368000001, 2024-09-21T15:34:13.181 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:34:13.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:34:13.184 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 76.009961032000007, 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.186 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:13.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:34:13.188 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:34:13.189 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.051485933999999, 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.190 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.191 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.192 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.193 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.050575077999994, 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.194 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:13.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:13.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:13.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.049609150999999, 2024-09-21T15:34:13.198 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.200 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.201 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:34:13.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.039405169000005, 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:13.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.207 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:15192 3.0 e58/50 3:002446f9:::rbd_data.11919eb7aeb1.000000000000004f:head v 58'302, mlcod=58'298)", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 42.259690822000003, 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032278039999999999, 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888712+0000", 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888728+0000", 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5973000000000001e-05 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888739+0000", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1039999999999999e-05 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888745+0000", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6049999999999997e-06 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888780+0000", 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5373999999999997e-05 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888800+0000", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9565e-05 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891899+0000", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030989469999999999 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891941+0000", 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2187000000000001e-05 2024-09-21T15:34:13.211 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62123 3.4 e68/47 3:22ecafc8:::rbd_data.1191e6bca1cd.00000000000000a8:head v 68'99, mlcod=65'97)", 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.556073799, 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032546599999999999, 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592327+0000", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592348+0000", 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1233000000000002e-05 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592397+0000", 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8353000000000003e-05 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592417+0000", 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0653e-05 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592769+0000", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035143300000000001 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592822+0000", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3137000000000003e-05 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595483+0000", 2024-09-21T15:34:13.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026611849999999999 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595585+0000", 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101462 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62269 3.0 e68/50 3:072ab387:::rbd_header.1191e6bca1cd:head v 68'364, mlcod=68'362)", 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.269863473000001, 2024-09-21T15:34:13.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037514879999999999, 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878538+0000", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878557+0000", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9173000000000001e-05 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878592+0000", 2024-09-21T15:34:13.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5429000000000002e-05 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878610+0000", 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7974000000000001e-05 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878698+0000", 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7293000000000006e-05 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878730+0000", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2175000000000003e-05 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882213+0000", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034834689999999999 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882292+0000", 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8453000000000003e-05 2024-09-21T15:34:13.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62282 3.0 e69/50 3:01c4b5a3:::rbd_data.1191e6bca1cd.00000000000000da:head v 69'365, mlcod=68'363)", 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.245738185, 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037099500000000001, 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:13.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902665+0000", 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902697+0000", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1680000000000002e-05 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902716+0000", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9145999999999999e-05 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902726+0000", 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0139000000000001e-05 2024-09-21T15:34:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902782+0000", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5986999999999998e-05 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902801+0000", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8300000000000001e-05 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906328+0000", 2024-09-21T15:34:13.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035275829999999999 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906376+0000", 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:102949 3.4 e72/47 3:221b15ed:::rbd_data.1191b30e35a1.0000000000000042:head v 72'106, mlcod=72'104)", 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.811824851000001, 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040397410000000003, 2024-09-21T15:34:13.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336577+0000", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336598+0000", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1659e-05 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336622+0000", 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3204e-05 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336633+0000", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0967e-05 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336686+0000", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3915999999999997e-05 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336712+0000", 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5060999999999999e-05 2024-09-21T15:34:13.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340579+0000", 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038670309999999999 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340619+0000", 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0080999999999998e-05 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.229 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:103104 3.9 e72/47 3:92ad05bc:::rbd_data.1191b30e35a1.000000000000002d:head v 72'107, mlcod=72'105)", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.670314486000001, 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034052539999999999, 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478088+0000", 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478140+0000", 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2240000000000001e-05 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478160+0000", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0176000000000001e-05 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478172+0000", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2133999999999999e-05 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478283+0000", 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110418 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478348+0000", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4735999999999998e-05 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481393+0000", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030456789999999999 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481495+0000", 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101309 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.6238438720000001, 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:13.238 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:13.240 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:13.240 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone28 from 0xa360993 (overlap 0x77690) to 0x3bb48 2024-09-21T15:34:13.240 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:13.235+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:13.325 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T15:34:13.340 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:13.336+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:13.417 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:13.417 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:13.417 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:13.418 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:13.418 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:13.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:13.440 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:13.440+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:13.541 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:13.539+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:13.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:13.600 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:13.625 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:13.640 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:13.640 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:13.640 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:13.640 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:13.641 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:13.643 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:13.639+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:13.649 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:13.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:13.682 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:13.711 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:13.711 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:13.741 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:13.742 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:13.744 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:13.743+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:13.780 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:13.780 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:13.810 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:13.810 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:13.850 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:13.850 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:13.850 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:13.847+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:13.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:13.946 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:13.943+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:14.047 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:14.043+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:14.053 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.053 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:14.148 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:14.144+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:14.241 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.241 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.249 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:14.248+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:14.253 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:14.350 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:14.347+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:14.406 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.406 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.419 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:14.451 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:14.447+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:14.523 INFO:teuthology.orchestra.run.smithi107.stdout:4083 write 0x28d328d thru 0x28df1a2 (0xbf16 bytes) 2024-09-21T15:34:14.539 INFO:teuthology.orchestra.run.smithi107.stdout:4085 read 0x8e8abba thru 0x8e924e3 (0x792a bytes) 2024-09-21T15:34:14.539 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.539 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.540 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:14.540 INFO:teuthology.orchestra.run.smithi107.stdout:4089 read 0x1b4ac93 thru 0x1b4c931 (0x1c9f bytes) 2024-09-21T15:34:14.548 INFO:teuthology.orchestra.run.smithi107.stdout:4090 trunc from 0xa360993 to 0x3a1081c 2024-09-21T15:34:14.552 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:14.551+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:14.558 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:14.604 INFO:teuthology.orchestra.run.smithi107.stdout:4091 trunc from 0x3a1081c to 0xe2e6628 2024-09-21T15:34:14.611 INFO:teuthology.orchestra.run.smithi107.stdout:4092 writesame 0x943dcba thru 0x94477db data_size 0x3(0x9b22 bytes) 2024-09-21T15:34:14.623 INFO:teuthology.orchestra.run.smithi107.stdout:4093 write 0xdb4972c thru 0xdb50403 (0x6cd8 bytes) 2024-09-21T15:34:14.628 INFO:teuthology.orchestra.run.smithi107.stdout:4096 write 0x8a76f35 thru 0x8a8180e (0xa8da bytes) 2024-09-21T15:34:14.645 INFO:teuthology.orchestra.run.smithi107.stdout:4098 trunc from 0xe2e6628 to 0x6afd774 2024-09-21T15:34:14.653 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:14.652+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:14.670 INFO:teuthology.orchestra.run.smithi107.stdout:4100 read 0x141f2b9 thru 0x142887d (0x95c5 bytes) 2024-09-21T15:34:14.671 INFO:teuthology.orchestra.run.smithi107.stdout:4101 punch from 0x44f7f37 to 0x44f92ea, (0x13b3 bytes) 2024-09-21T15:34:14.671 INFO:teuthology.orchestra.run.smithi107.stdout:4102 read 0x2042928 thru 0x20512b7 (0xe990 bytes) 2024-09-21T15:34:14.672 INFO:teuthology.orchestra.run.smithi107.stdout:4103 write 0x3029bb9 thru 0x3039624 (0xfa6c bytes) 2024-09-21T15:34:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:4104 read 0x2878bf9 thru 0x2880f11 (0x8319 bytes) 2024-09-21T15:34:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:4105 read 0x51bcb21 thru 0x51c60d3 (0x95b3 bytes) 2024-09-21T15:34:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:4106 trunc from 0x6afd774 to 0x65c296b 2024-09-21T15:34:14.686 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.686 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.692 INFO:teuthology.orchestra.run.smithi107.stdout:4107 read 0x15cb7bb thru 0x15d67a9 (0xafef bytes) 2024-09-21T15:34:14.692 INFO:teuthology.orchestra.run.smithi107.stdout:4108 punch from 0x2167ec3 to 0x216b25b, (0x3398 bytes) 2024-09-21T15:34:14.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:14.709 INFO:teuthology.orchestra.run.smithi107.stdout:4109 writesame 0x9695201 thru 0x969599e data_size 0x3(0x79e bytes) 2024-09-21T15:34:14.710 INFO:teuthology.orchestra.run.smithi107.stdout:4110 read 0x1e8f059 thru 0x1e9107e (0x2026 bytes) 2024-09-21T15:34:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:4111 trunc from 0x969599f to 0x2038c64 2024-09-21T15:34:14.740 INFO:teuthology.orchestra.run.smithi107.stdout:4112 punch from 0xb04c2e to 0xb0e5a3, (0x9975 bytes) 2024-09-21T15:34:14.740 INFO:teuthology.orchestra.run.smithi107.stdout:4113 read 0x16bb86c thru 0x16bf2c0 (0x3a55 bytes) 2024-09-21T15:34:14.740 INFO:teuthology.orchestra.run.smithi107.stdout:4115 punch from 0xc8614e to 0xc92f38, (0xcdea bytes) 2024-09-21T15:34:14.753 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:14.751+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:14.759 INFO:teuthology.orchestra.run.smithi107.stdout:4116 trunc from 0x2038c64 to 0x95fdeec 2024-09-21T15:34:14.766 INFO:teuthology.orchestra.run.smithi107.stdout:4117 read 0x2fd1ec2 thru 0x2fda56c (0x86ab bytes) 2024-09-21T15:34:14.767 INFO:teuthology.orchestra.run.smithi107.stdout:4119 read 0x406cfab thru 0x4079d87 (0xcddd bytes) 2024-09-21T15:34:14.767 INFO:teuthology.orchestra.run.smithi107.stdout:4121 clone 30 order 25 su 2097152 sc 13 2024-09-21T15:34:14.854 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:14.851+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.859 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:14.873 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:14.955 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:14.950+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:14.993 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:15.056 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:15.054+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:15.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:34:15.134 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:15.135 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:15.156 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:15.154+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:15.199 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone29 from 0x95fdeec (overlap 0x2038c64) to 0x135543c 2024-09-21T15:34:15.257 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:15.256+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:15.284 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:15.285 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.300 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:15.338 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T15:34:15.344 INFO:teuthology.orchestra.run.smithi107.stdout:4122 read 0x774c11c thru 0x775855a (0xc43f bytes) 2024-09-21T15:34:15.346 INFO:teuthology.orchestra.run.smithi107.stdout:4124 read 0x50d54a5 thru 0x50e5261 (0xfdbd bytes) 2024-09-21T15:34:15.347 INFO:teuthology.orchestra.run.smithi107.stdout:4125 read 0x3cf1a20 thru 0x3cfc21f (0xa800 bytes) 2024-09-21T15:34:15.349 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.349 INFO:teuthology.orchestra.run.smithi107.stdout:4127 read 0x8b76edf thru 0x8b80482 (0x95a4 bytes) 2024-09-21T15:34:15.349 INFO:teuthology.orchestra.run.smithi107.stdout:4128 writesame 0x1db65e6 thru 0x1dbaa6b data_size 0x31(0x4486 bytes) 2024-09-21T15:34:15.364 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.365 INFO:teuthology.orchestra.run.smithi107.stdout:4133 write 0x84803be thru 0x84899ff (0x9642 bytes) 2024-09-21T15:34:15.368 INFO:teuthology.orchestra.run.smithi107.stdout:4134 read 0x1bcc6cb thru 0x1bd7b19 (0xb44f bytes) 2024-09-21T15:34:15.369 INFO:teuthology.orchestra.run.smithi107.stdout:4135 punch from 0x28acc60 to 0x28bb577, (0xe917 bytes) 2024-09-21T15:34:15.369 INFO:teuthology.orchestra.run.smithi107.stdout:4137 write 0x230efa8 thru 0x231281f (0x3878 bytes) 2024-09-21T15:34:15.375 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.375 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.375 INFO:teuthology.orchestra.run.smithi107.stdout:4140 read 0x9c0370 thru 0x9cc1d4 (0xbe65 bytes) 2024-09-21T15:34:15.375 INFO:teuthology.orchestra.run.smithi107.stdout:4142 write 0x25065b5 thru 0x2511144 (0xab90 bytes) 2024-09-21T15:34:15.384 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.384 INFO:teuthology.orchestra.run.smithi107.stdout:4144 read 0x48c6142 thru 0x48d2672 (0xc531 bytes) 2024-09-21T15:34:15.384 INFO:teuthology.orchestra.run.smithi107.stdout:4145 read 0x2adf8cd thru 0x2aecf70 (0xd6a4 bytes) 2024-09-21T15:34:15.384 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.385 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:15.384+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:15.396 INFO:teuthology.orchestra.run.smithi107.stdout:4148 write 0xb037f9e thru 0xb03b44d (0x34b0 bytes) 2024-09-21T15:34:15.403 INFO:teuthology.orchestra.run.smithi107.stdout:4151 read 0x2207b93 thru 0x220ce67 (0x52d5 bytes) 2024-09-21T15:34:15.404 INFO:teuthology.orchestra.run.smithi107.stdout:4153 punch from 0x99fdbcb to 0x9a05df5, (0x822a bytes) 2024-09-21T15:34:15.404 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.404 INFO:teuthology.orchestra.run.smithi107.stdout:4157 read 0x4ad2107 thru 0x4ada429 (0x8323 bytes) 2024-09-21T15:34:15.405 INFO:teuthology.orchestra.run.smithi107.stdout:4158 write 0x303198b thru 0x303dfbf (0xc635 bytes) 2024-09-21T15:34:15.420 INFO:teuthology.orchestra.run.smithi107.stdout:4159 write 0xb6e4e9d thru 0xb6e5b54 (0xcb8 bytes) 2024-09-21T15:34:15.431 INFO:teuthology.orchestra.run.smithi107.stdout:4161 read 0x8ce4a7c thru 0x8ce534e (0x8d3 bytes) 2024-09-21T15:34:15.431 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.431 INFO:teuthology.orchestra.run.smithi107.stdout:4163 write 0x6091dc thru 0x60b959 (0x277e bytes) 2024-09-21T15:34:15.438 INFO:teuthology.orchestra.run.smithi107.stdout:4165 punch from 0x7d3e1ac to 0x7d4dc21, (0xfa75 bytes) 2024-09-21T15:34:15.448 INFO:teuthology.orchestra.run.smithi107.stdout:4166 writesame 0xc20eed4 thru 0xc2147ea data_size 0x1(0x5917 bytes) 2024-09-21T15:34:15.459 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:15.454+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:15.465 INFO:teuthology.orchestra.run.smithi107.stdout:4168 writesame 0xc8919c7 thru 0xc892fc1 data_size 0x1(0x15fb bytes) 2024-09-21T15:34:15.466 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:15.466 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.470 INFO:teuthology.orchestra.run.smithi107.stdout:4169 punch from 0x15c8b02 to 0x15d3535, (0xaa33 bytes) 2024-09-21T15:34:15.471 INFO:teuthology.orchestra.run.smithi107.stdout:4170 read 0x77d50ba thru 0x77dfb8d (0xaad4 bytes) 2024-09-21T15:34:15.471 INFO:teuthology.orchestra.run.smithi107.stdout:4172 write 0xcd679c thru 0xce3306 (0xcb6b bytes) 2024-09-21T15:34:15.474 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:15.476 INFO:teuthology.orchestra.run.smithi107.stdout:4174 trunc from 0xc892fc2 to 0xb759e54 2024-09-21T15:34:15.483 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:15.497 INFO:teuthology.orchestra.run.smithi107.stdout:4176 write 0x7854892 thru 0x786314a (0xe8b9 bytes) 2024-09-21T15:34:15.499 INFO:teuthology.orchestra.run.smithi107.stdout:4177 write 0x707791 thru 0x7167e5 (0xf055 bytes) 2024-09-21T15:34:15.511 INFO:teuthology.orchestra.run.smithi107.stdout:4179 write 0xb8ecb4a thru 0xb8ecd8c (0x243 bytes) 2024-09-21T15:34:15.513 INFO:teuthology.orchestra.run.smithi107.stdout:4180 punch from 0x928a774 to 0x9296941, (0xc1cd bytes) 2024-09-21T15:34:15.513 INFO:teuthology.orchestra.run.smithi107.stdout:4181 clone 31 order 18 su 131072 sc 10 2024-09-21T15:34:15.560 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:15.558+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:15.651 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:15.651 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.661 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:15.660+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:15.665 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:15.761 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:15.760+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:15.830 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:15.830 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:15.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:15.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:15.880 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:15.878+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:15.963 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:15.960+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:15.988 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:15.988 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:16.005 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:16.005 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:16.006 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.007 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:16.007 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:16.007 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:16.021 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:16.021 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:16.037 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:16.060 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:16.060 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:16.064 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:16.062+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:16.078 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:34:16.083 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:16.084 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:16.106 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:16.127 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:16.127 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:16.150 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.150 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:16.150 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:16.150 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:16.150 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:16.165 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:16.164+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:16.207 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:16.207 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:16.207 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:34:16.207 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:34:16.246 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone30 2024-09-21T15:34:16.266 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:16.264+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 77.623312515999999, 2024-09-21T15:34:16.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:34:16.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.685492995999994, 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:34:16.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:34:16.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:34:16.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:34:16.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:34:16.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124484 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2975744~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9618634860000004, 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043060238000000001, 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124484 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371116+0000", 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371118+0000", 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9460000000000001e-06 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371123+0000", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3700000000000003e-06 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371127+0000", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0280000000000002e-06 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414044+0000", 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042916081000000002 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414115+0000", 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1607999999999996e-05 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414177+0000", 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1617000000000003e-05 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124485 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2985984~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.349 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9618489559999999, 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043229165, 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124485 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371133+0000", 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8899999999999999e-06 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371138+0000", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1359999999999996e-06 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371147+0000", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8340000000000005e-06 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414210+0000", 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043063203000000001 2024-09-21T15:34:16.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414291+0000", 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1428999999999996e-05 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414360+0000", 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9033000000000001e-05 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.353 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124486 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2996224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9618298559999996, 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043374530000000001, 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124486 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371152+0000", 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.888e-06 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371157+0000", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2560000000000002e-06 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371161+0000", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0899999999999998e-06 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414392+0000", 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043231251999999998 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414461+0000", 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8369999999999998e-05 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414525+0000", 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4046000000000003e-05 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124487 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3006464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9618154380000004, 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043541654999999999, 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124487 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371164+0000", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371166+0000", 2024-09-21T15:34:16.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7460000000000001e-06 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371171+0000", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0719999999999999e-06 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371175+0000", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0659999999999997e-06 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414556+0000", 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043381182999999997 2024-09-21T15:34:16.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414636+0000", 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9809999999999997e-05 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414706+0000", 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0134000000000003e-05 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.361 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124488 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3016704~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9618015739999999, 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692364999999997, 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124488 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371180+0000", 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.734e-06 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371185+0000", 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371189+0000", 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.912e-06 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414738+0000", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043548877 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414809+0000", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1675999999999996e-05 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414871+0000", 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1477999999999997e-05 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124489 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3026944~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9617876379999997, 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043866982999999998, 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124489 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371194+0000", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371199+0000", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0220000000000004e-06 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371203+0000", 2024-09-21T15:34:16.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.941e-06 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414912+0000", 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043709439000000003 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414992+0000", 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9572000000000003e-05 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415059+0000", 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7539000000000005e-05 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124490 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3037184~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9617737569999996, 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044018785999999997, 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.370 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124490 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371208+0000", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8080000000000001e-06 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371213+0000", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371217+0000", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0300000000000004e-06 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415092+0000", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043875441000000001 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415163+0000", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1026999999999996e-05 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.373 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415225+0000", 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1793999999999998e-05 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124491 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3047424~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9617596930000003, 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044189697999999999, 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.374 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124491 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371230+0000", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0068000000000001e-05 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371244+0000", 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.395e-05 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371251+0000", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1960000000000002e-06 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415258+0000", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044007019000000001 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415339+0000", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1036000000000001e-05 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.377 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415410+0000", 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0784999999999994e-05 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124492 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3057664~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9617042730000005, 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044296730999999999, 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124492 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371275+0000", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371279+0000", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5260000000000002e-06 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371291+0000", 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2098e-05 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371307+0000", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6787999999999999e-05 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415440+0000", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044132678000000002 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415510+0000", 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9999999999999994e-05 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415572+0000", 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2333000000000001e-05 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124495 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3088384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9600139419999998, 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043138292000000002, 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.382 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124495 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:16.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372965+0000", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372969+0000", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0119999999999999e-06 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372982+0000", 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2517000000000001e-05 2024-09-21T15:34:16.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372991+0000", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4180000000000003e-06 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415952+0000", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042960920999999999 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416034+0000", 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1464999999999993e-05 2024-09-21T15:34:16.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416104+0000", 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0635000000000004e-05 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124496 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3098624~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9599783789999998, 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043270034999999998, 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124496 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:16.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373001+0000", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373005+0000", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7340000000000002e-06 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373020+0000", 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5622000000000001e-05 2024-09-21T15:34:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373029+0000", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0240000000000003e-06 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416137+0000", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043107686999999999 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416206+0000", 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999995e-05 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416272+0000", 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5283999999999994e-05 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124497 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3108864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9599414289999997, 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043416800999999998, 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124497 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373038+0000", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373041+0000", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5880000000000002e-06 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373052+0000", 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.057e-05 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373060+0000", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8720000000000002e-06 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416303+0000", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043243216000000001 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416386+0000", 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2712000000000004e-05 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416455+0000", 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9535000000000004e-05 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124498 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3119104~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9599111230000004, 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043553477, 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124498 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373068+0000", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373072+0000", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3220000000000001e-06 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373082+0000", 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0368e-05 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373100+0000", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8309e-05 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416488+0000", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043387362999999998 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416560+0000", 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2568e-05 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416622+0000", 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2212999999999995e-05 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124499 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3129344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9598723339999999, 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692511000000003, 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124499 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373107+0000", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373111+0000", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373122+0000", 2024-09-21T15:34:16.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1590000000000001e-05 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373131+0000", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3519999999999996e-06 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416653+0000", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043521988999999997 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416733+0000", 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9479000000000003e-05 2024-09-21T15:34:16.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416800+0000", 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7218999999999995e-05 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124500 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3139584~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9598305839999997, 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043824758999999998, 2024-09-21T15:34:16.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124500 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373152+0000", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.58e-06 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373166+0000", 2024-09-21T15:34:16.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3835999999999999e-05 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373176+0000", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5019999999999998e-06 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416833+0000", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043656815000000002 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416911+0000", 2024-09-21T15:34:16.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8788999999999994e-05 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416974+0000", 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2854999999999994e-05 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124521 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4077568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9553572629999998, 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043139638000000001, 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124521 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377622+0000", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377626+0000", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377638+0000", 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2216e-05 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377653+0000", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5114e-05 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420629+0000", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042976331 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420699+0000", 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9349000000000003e-05 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420762+0000", 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3812000000000005e-05 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124522 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3997696~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9553181569999998, 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043278491000000002, 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.410 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124522 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377661+0000", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377665+0000", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377675+0000", 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0621999999999999e-05 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.412 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377695+0000", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9592000000000001e-05 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420796+0000", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043101511000000002 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420877+0000", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0694999999999999e-05 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.413 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420940+0000", 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3170999999999995e-05 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124523 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4087808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 7.9552024580000005, 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043336070999999997, 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.414 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124523 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377702+0000", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999242 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377782+0000", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0505000000000003e-05 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377790+0000", 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2740000000000007e-06 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377796+0000", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420979+0000", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043182812000000001 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421048+0000", 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9449999999999994e-05 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421114+0000", 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5278999999999996e-05 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.418 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:16.420 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:16.414+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:16.473 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T15:34:16.510 INFO:teuthology.orchestra.run.smithi107.stdout:4183 punch from 0x479f275 to 0x47a5b32, (0x68bd bytes) 2024-09-21T15:34:16.520 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:16.516+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 84, 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:34:16.068135+0000", 2024-09-21T15:34:16.524 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:33:20.030364+0000", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5804032, 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "nodeep-scrub", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:34:16.525 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "luminous", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:34:16.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:34:16.527 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:16.528 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:16.529 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:16.530 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:34:16.531 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:34:16.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:34:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:34:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:16.535 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:16.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:34:16.537 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:16.538 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:34:16.539 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "84", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:16.540 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 34, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 84, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:16.541 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:16.542 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:34:16.543 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.544 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.545 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.547 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.548 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.549 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 56, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.550 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:34:16.554 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.555 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:34:16.556 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:34:16.557 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.558 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.75054931640625, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 57, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.561 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.562 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.563 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.564 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.566 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.8086395263671875, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 59, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.568 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:34:16.569 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:34:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.571 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.9112701416015625, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:34:16.572 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.577 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 53, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:34:16.579 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:34:16.580 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:34:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:34:16.582 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0.71441650390625, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:16.583 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 68, 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.587 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.588 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:34:16.589 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.590 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:34:16.591 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:16.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [ 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.10", 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: 7, 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: 5 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:34:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:34:16.595 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [], 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:34:16.596 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:34:16.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.597 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.597 INFO:teuthology.orchestra.run.smithi107.stdout:4184 trunc from 0xb8ecd8d to 0x7260ba2 2024-09-21T15:34:16.599 INFO:tasks.thrashosds.thrasher:j is {'epoch': 84, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:34:16.068135+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:33:20.030364+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 0.75054931640625, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 0.8086395263671875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 0.9112701416015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 0.71441650390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.10', 'osds': [7, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:34:16.599 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:16.599 INFO:teuthology.orchestra.run.smithi107.stdout:4185 writesame 0x5fb98ec thru 0x5fc7a99 data_size 0x1(0xe1ae bytes) 2024-09-21T15:34:16.599 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.599 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:16.600 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:16.600 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:16.621 INFO:teuthology.orchestra.run.smithi107.stdout:4187 writesame 0x242a7c9 thru 0x242be1a data_size 0x1(0x1652 bytes) 2024-09-21T15:34:16.622 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:16.618+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:16.633 INFO:teuthology.orchestra.run.smithi107.stdout:4189 write 0xe3b451 thru 0xe414ab (0x605b bytes) 2024-09-21T15:34:16.642 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:16.642 INFO:teuthology.orchestra.run.smithi107.stdout:4191 punch from 0x47ec3c5 to 0x47f2a9a, (0x66d5 bytes) 2024-09-21T15:34:16.666 INFO:teuthology.orchestra.run.smithi107.stdout:4193 writesame 0xedf19dd thru 0xedfa18f data_size 0x1(0x87b3 bytes) 2024-09-21T15:34:16.666 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:16.666 INFO:teuthology.orchestra.run.smithi107.stdout:4195 writesame 0x2efab89 thru 0x2f0695c data_size 0x2(0xbdd4 bytes) 2024-09-21T15:34:16.681 INFO:teuthology.orchestra.run.smithi107.stdout:4196 writesame 0xa49116e thru 0xa4a0f7e data_size 0x1(0xfe11 bytes) 2024-09-21T15:34:16.722 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:16.718+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.752 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:16.752 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:16.752 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:4198 read 0x6e3fa2 thru 0x6e9228 (0x5287 bytes) 2024-09-21T15:34:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:4199 read 0x72be2f4 thru 0x72bfaf7 (0x1804 bytes) 2024-09-21T15:34:16.755 INFO:teuthology.orchestra.run.smithi107.stdout:4200 write 0x8deffd2 thru 0x8df496d (0x499c bytes) 2024-09-21T15:34:16.762 INFO:teuthology.orchestra.run.smithi107.stdout:4201 read 0x8c5298c thru 0x8c59780 (0x6df5 bytes) 2024-09-21T15:34:16.762 INFO:teuthology.orchestra.run.smithi107.stdout:4202 write 0x2541a3c thru 0x2550cf2 (0xf2b7 bytes) 2024-09-21T15:34:16.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:16.769 INFO:teuthology.orchestra.run.smithi107.stdout:4203 writesame 0x51796c thru 0x51c87f data_size 0x2a(0x4f14 bytes) 2024-09-21T15:34:16.776 INFO:teuthology.orchestra.run.smithi107.stdout:4204 writesame 0x91ca88d thru 0x91d4861 data_size 0x17(0x9fd5 bytes) 2024-09-21T15:34:16.783 INFO:teuthology.orchestra.run.smithi107.stdout:4205 read 0x179393f thru 0x17a382a (0xfeec bytes) 2024-09-21T15:34:16.784 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:16.804 INFO:teuthology.orchestra.run.smithi107.stdout:4208 trunc from 0xedfa190 to 0xbd293d 2024-09-21T15:34:16.822 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:16.818+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:16.857 INFO:teuthology.orchestra.run.smithi107.stdout:4209 write 0x3508662 thru 0x351211f (0x9abe bytes) 2024-09-21T15:34:16.860 INFO:teuthology.orchestra.run.smithi107.stdout:4210 read 0x30092a8 thru 0x3010eb2 (0x7c0b bytes) 2024-09-21T15:34:16.862 INFO:teuthology.orchestra.run.smithi107.stdout:4211 trunc from 0x3512120 to 0x4352e8c 2024-09-21T15:34:16.872 INFO:teuthology.orchestra.run.smithi107.stdout:4213 read 0x3b57e4e thru 0x3b60dbd (0x8f70 bytes) 2024-09-21T15:34:16.872 INFO:teuthology.orchestra.run.smithi107.stdout:4214 read 0x1007c66 thru 0x101706e (0xf409 bytes) 2024-09-21T15:34:16.891 INFO:teuthology.orchestra.run.smithi107.stdout:4215 write 0xb595a7d thru 0xb59e807 (0x8d8b bytes) 2024-09-21T15:34:16.897 INFO:teuthology.orchestra.run.smithi107.stdout:4216 write 0x3f37401 thru 0x3f408d9 (0x94d9 bytes) 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.902 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 61.778105732999997, 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:16.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:16.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.908 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.909 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.33594952, 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.911 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.912 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:16.913 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.335924773999999, 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.914 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:16.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.916 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:16.917 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.335879824000003, 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.918 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.920 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.332023165999999, 2024-09-21T15:34:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331978806000002, 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331925456999997, 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331894063999997, 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331848338, 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331818964999997, 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331794404999997, 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.331767767999999, 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327929201000003, 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327899381000002, 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327873851, 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327848144999997, 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327798104999999, 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327763304999998, 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.327738656000001, 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.323688249, 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:16.986 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:16.987 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:16.988 INFO:teuthology.orchestra.run.smithi107.stdout:4218 read 0x309e6bd thru 0x30a9570 (0xaeb4 bytes) 2024-09-21T15:34:16.988 INFO:teuthology.orchestra.run.smithi107.stdout:4219 clone 32 order 22 su 1048576 sc 8 2024-09-21T15:34:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:16.989 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:16.990 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-21T15:34:16.076640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":680937933,"num_objects":1108,"num_object_clones":239,"num_object_copies":2216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1108,"num_whiteouts":451,"num_read":159083,"num_read_kb":254751,"num_write":11246,"num_write_kb":269058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":5997309,"num_keys_recovered":35,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8829,"ondisk_log_size":8829,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405140,"kb_used_data":143944,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754569580,"statfs":{"total":773094113280,"available":772679249920,"internally_reserved":0,"allocated":147398656,"data_stored":167869299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,13,0,16,35,107,68],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30472895,"num_objects":21,"num_object_clones":12,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":11,"num_read":10548,"num_read_kb":10029,"num_write":742,"num_write_kb":26979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.039049"},"pg_stats":[{"pgid":"3.1f","version":"82'321","reported_seq":2885,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.677360+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:13.677360+0000","last_peered":"2024-09-21T15:34:13.677360+0000","last_clean":"2024-09-21T15:34:13.677360+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:13.677360+0000","last_undegraded":"2024-09-21T15:34:13.677360+0000","last_fullsized":"2024-09-21T15:34:13.677360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":19777423,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":3718,"num_read_kb":5198,"num_write":476,"num_write_kb":2011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"78'249","reported_seq":3584,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.679655+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:13.679655+0000","last_peered":"2024-09-21T15:34:13.679655+0000","last_clean":"2024-09-21T15:34:13.679655+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:13.679655+0000","last_undegraded":"2024-09-21T15:34:13.679655+0000","last_fullsized":"2024-09-21T15:34:13.679655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":23958426,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":3951,"num_read_kb":12739,"num_write":393,"num_write_kb":25131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"81'108","reported_seq":352,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.065734+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:13.065734+0000","last_peered":"2024-09-21T15:34:13.065734+0000","last_clean":"2024-09-21T15:34:13.065734+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:13.065734+0000","last_undegraded":"2024-09-21T15:34:13.065734+0000","last_fullsized":"2024-09-21T15:34:13.065734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":16183579,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":220,"num_read_kb":3405,"num_write":163,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"82'178","reported_seq":1228,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.315734+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:13.315734+0000","last_peered":"2024-09-21T15:34:13.315734+0000","last_clean":"2024-09-21T15:34:13.315734+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:13.315734+0000","last_undegraded":"2024-09-21T15:34:13.315734+0000","last_fullsized":"2024-09-21T15:34:13.315734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46730960,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":1149,"num_read_kb":4738,"num_write":188,"num_write_kb":4798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'278","reported_seq":6071,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.677541+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:13.677541+0000","last_peered":"2024-09-21T15:34:13.677541+0000","last_clean":"2024-09-21T15:34:13.677541+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:13.677541+0000","last_undegraded":"2024-09-21T15:34:13.677541+0000","last_fullsized":"2024-09-21T15:34:13.677541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":12448712,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":6490,"num_read_kb":7351,"num_write":279,"num_write_kb":3796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"78'142","reported_seq":1897,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.321593+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:13.321593+0000","last_peered":"2024-09-21T15:34:13.321593+0000","last_clean":"2024-09-21T15:34:13.321593+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:13.321593+0000","last_undegraded":"2024-09-21T15:34:13.321593+0000","last_fullsized":"2024-09-21T15:34:13.321593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":9969937,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":1750,"num_read_kb":7549,"num_write":138,"num_write_kb":2747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"82'533","reported_seq":33151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.191970+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:13.191970+0000","last_peered":"2024-09-21T15:34:13.191970+0000","last_clean":"2024-09-21T15:34:13.191970+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:13.191970+0000","last_undegraded":"2024-09-21T15:34:13.191970+0000","last_fullsized":"2024-09-21T15:34:13.191970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":47948976,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":34530,"num_read_kb":18664,"num_write":772,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'362","reported_seq":757,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.051941+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:13.051941+0000","last_peered":"2024-09-21T15:34:13.051941+0000","last_clean":"2024-09-21T15:34:13.051941+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:13.051941+0000","last_undegraded":"2024-09-21T15:34:13.051941+0000","last_fullsized":"2024-09-21T15:34:13.051941+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":14817896,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":1920,"num_read_kb":3306,"num_write":525,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"81'500","reported_seq":1675,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.805086+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:10.805086+0000","last_peered":"2024-09-21T15:34:10.805086+0000","last_clean":"2024-09-21T15:34:10.805086+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:10.805086+0000","last_undegraded":"2024-09-21T15:34:10.805086+0000","last_fullsized":"2024-09-21T15:34:10.805086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":23580971,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":3624,"num_read_kb":9288,"num_write":778,"num_write_kb":5720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"78'275","reported_seq":3729,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.676876+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:13.676876+0000","last_peered":"2024-09-21T15:34:13.676876+0000","last_clean":"2024-09-21T15:34:13.676876+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:13.676876+0000","last_undegraded":"2024-09-21T15:34:13.676876+0000","last_fullsized":"2024-09-21T15:34:13.676876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27601754,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":4948,"num_read_kb":8201,"num_write":283,"num_write_kb":3555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"81'204","reported_seq":2613,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.680291+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:34:13.680291+0000","last_peered":"2024-09-21T15:34:13.680291+0000","last_clean":"2024-09-21T15:34:13.680291+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:34:13.680291+0000","last_undegraded":"2024-09-21T15:34:13.680291+0000","last_fullsized":"2024-09-21T15:34:13.680291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":6721825,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":2633,"num_read_kb":5477,"num_write":239,"num_write_kb":7464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"82'409","reported_seq":1478,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.192949+0000","last_change":"2024-09-21T15:32:48.726261+0000","last_active":"2024-09-21T15:34:13.192949+0000","last_peered":"2024-09-21T15:34:13.192949+0000","last_clean":"2024-09-21T15:34:13.192949+0000","last_became_active":"2024-09-21T15:32:46.731389+0000","last_became_peered":"2024-09-21T15:32:46.731389+0000","last_unstale":"2024-09-21T15:34:13.192949+0000","last_undegraded":"2024-09-21T15:34:13.192949+0000","last_fullsized":"2024-09-21T15:34:13.192949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:02.678176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":27544870,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":3019,"num_read_kb":5077,"num_write":491,"num_write_kb":11319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.076954+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:34:10.076954+0000","last_peered":"2024-09-21T15:34:10.076954+0000","last_clean":"2024-09-21T15:34:10.076954+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:34:10.076954+0000","last_undegraded":"2024-09-21T15:34:10.076954+0000","last_fullsized":"2024-09-21T15:34:10.076954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"78'208","reported_seq":1316,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.052763+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:13.052763+0000","last_peered":"2024-09-21T15:34:13.052763+0000","last_clean":"2024-09-21T15:34:13.052763+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:13.052763+0000","last_undegraded":"2024-09-21T15:34:13.052763+0000","last_fullsized":"2024-09-21T15:34:13.052763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":12039443,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":2654,"num_read_kb":5153,"num_write":197,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.041677+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:10.041677+0000","last_peered":"2024-09-21T15:34:10.041677+0000","last_clean":"2024-09-21T15:34:10.041677+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:10.041677+0000","last_undegraded":"2024-09-21T15:34:10.041677+0000","last_fullsized":"2024-09-21T15:34:10.041677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"81'154","reported_seq":1115,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:12.072890+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:12.072890+0000","last_peered":"2024-09-21T15:34:12.072890+0000","last_clean":"2024-09-21T15:34:12.072890+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:12.072890+0000","last_undegraded":"2024-09-21T15:34:12.072890+0000","last_fullsized":"2024-09-21T15:34:12.072890+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":32374075,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":1324,"num_read_kb":20773,"num_write":175,"num_write_kb":17131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.131673+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:10.131673+0000","last_peered":"2024-09-21T15:34:10.131673+0000","last_clean":"2024-09-21T15:34:10.131673+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:10.131673+0000","last_undegraded":"2024-09-21T15:34:10.131673+0000","last_fullsized":"2024-09-21T15:34:10.131673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"78'140","reported_seq":655,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.052549+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:13.052549+0000","last_peered":"2024-09-21T15:34:13.052549+0000","last_clean":"2024-09-21T15:34:13.052549+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:13.052549+0000","last_undegraded":"2024-09-21T15:34:13.052549+0000","last_fullsized":"2024-09-21T15:34:13.052549+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":12859316,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":776,"num_read_kb":3937,"num_write":147,"num_write_kb":2555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.041265+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:10.041265+0000","last_peered":"2024-09-21T15:34:10.041265+0000","last_clean":"2024-09-21T15:34:10.041265+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:10.041265+0000","last_undegraded":"2024-09-21T15:34:10.041265+0000","last_fullsized":"2024-09-21T15:34:10.041265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"78'353","reported_seq":711,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.303105+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:13.303105+0000","last_peered":"2024-09-21T15:34:13.303105+0000","last_clean":"2024-09-21T15:34:13.303105+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:13.303105+0000","last_undegraded":"2024-09-21T15:34:13.303105+0000","last_fullsized":"2024-09-21T15:34:13.303105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":10089065,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":1425,"num_read_kb":4708,"num_write":464,"num_write_kb":3660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'150","reported_seq":2636,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.681311+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:13.681311+0000","last_peered":"2024-09-21T15:34:13.681311+0000","last_clean":"2024-09-21T15:34:13.681311+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:13.681311+0000","last_undegraded":"2024-09-21T15:34:13.681311+0000","last_fullsized":"2024-09-21T15:34:13.681311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":17294908,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":2464,"num_read_kb":7788,"num_write":180,"num_write_kb":3829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.039862+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:10.039862+0000","last_peered":"2024-09-21T15:34:10.039862+0000","last_clean":"2024-09-21T15:34:10.039862+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:10.039862+0000","last_undegraded":"2024-09-21T15:34:10.039862+0000","last_fullsized":"2024-09-21T15:34:10.039862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"81'451","reported_seq":1330,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.051936+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:13.051936+0000","last_peered":"2024-09-21T15:34:13.051936+0000","last_clean":"2024-09-21T15:34:13.051936+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:13.051936+0000","last_undegraded":"2024-09-21T15:34:13.051936+0000","last_fullsized":"2024-09-21T15:34:13.051936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25050295,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":2700,"num_read_kb":4357,"num_write":594,"num_write_kb":10134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.063593+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:10.063593+0000","last_peered":"2024-09-21T15:34:10.063593+0000","last_clean":"2024-09-21T15:34:10.063593+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:10.063593+0000","last_undegraded":"2024-09-21T15:34:10.063593+0000","last_fullsized":"2024-09-21T15:34:10.063593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'373","reported_seq":844,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.292555+0000","last_change":"2024-09-21T15:33:29.748980+0000","last_active":"2024-09-21T15:34:13.292555+0000","last_peered":"2024-09-21T15:34:13.292555+0000","last_clean":"2024-09-21T15:34:13.292555+0000","last_became_active":"2024-09-21T15:33:29.747834+0000","last_became_peered":"2024-09-21T15:33:29.747834+0000","last_unstale":"2024-09-21T15:34:13.292555+0000","last_undegraded":"2024-09-21T15:34:13.292555+0000","last_fullsized":"2024-09-21T15:34:13.292555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:19:53.405024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":18188363,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":1755,"num_read_kb":4046,"num_write":594,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":399716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.131748+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:10.131748+0000","last_peered":"2024-09-21T15:34:10.131748+0000","last_clean":"2024-09-21T15:34:10.131748+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:10.131748+0000","last_undegraded":"2024-09-21T15:34:10.131748+0000","last_fullsized":"2024-09-21T15:34:10.131748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"82'144","reported_seq":539,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.472690+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:13.472690+0000","last_peered":"2024-09-21T15:34:13.472690+0000","last_clean":"2024-09-21T15:34:13.472690+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:13.472690+0000","last_undegraded":"2024-09-21T15:34:13.472690+0000","last_fullsized":"2024-09-21T15:34:13.472690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":6833842,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":624,"num_read_kb":3342,"num_write":123,"num_write_kb":1364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.039807+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:10.039807+0000","last_peered":"2024-09-21T15:34:10.039807+0000","last_clean":"2024-09-21T15:34:10.039807+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:10.039807+0000","last_undegraded":"2024-09-21T15:34:10.039807+0000","last_fullsized":"2024-09-21T15:34:10.039807+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.042084+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:34:10.042084+0000","last_peered":"2024-09-21T15:34:10.042084+0000","last_clean":"2024-09-21T15:34:10.042084+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:34:10.042084+0000","last_undegraded":"2024-09-21T15:34:10.042084+0000","last_fullsized":"2024-09-21T15:34:10.042084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"79'442","reported_seq":1185,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T15:34:10.040982+0000","last_change":"2024-09-21T15:33:29.749882+0000","last_active":"2024-09-21T15:34:10.040982+0000","last_peered":"2024-09-21T15:34:10.040982+0000","last_clean":"2024-09-21T15:34:10.040982+0000","last_became_active":"2024-09-21T15:33:29.748534+0000","last_became_peered":"2024-09-21T15:33:29.748534+0000","last_unstale":"2024-09-21T15:34:10.040982+0000","last_undegraded":"2024-09-21T15:34:10.040982+0000","last_fullsized":"2024-09-21T15:34:10.040982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:33.004724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":11292668,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2568,"num_read_kb":6801,"num_write":635,"num_write_kb":5310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":786567,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'221","reported_seq":1190,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.260880+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:13.260880+0000","last_peered":"2024-09-21T15:34:13.260880+0000","last_clean":"2024-09-21T15:34:13.260880+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:13.260880+0000","last_undegraded":"2024-09-21T15:34:13.260880+0000","last_fullsized":"2024-09-21T15:34:13.260880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24391062,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":1825,"num_read_kb":6485,"num_write":220,"num_write_kb":3115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"82'226","reported_seq":3476,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.678677+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:13.678677+0000","last_peered":"2024-09-21T15:34:13.678677+0000","last_clean":"2024-09-21T15:34:13.678677+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:13.678677+0000","last_undegraded":"2024-09-21T15:34:13.678677+0000","last_fullsized":"2024-09-21T15:34:13.678677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":20274036,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":21,"num_read":3509,"num_read_kb":11479,"num_write":253,"num_write_kb":4848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"78'206","reported_seq":1107,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.052193+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:13.052193+0000","last_peered":"2024-09-21T15:34:13.052193+0000","last_clean":"2024-09-21T15:34:13.052193+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:13.052193+0000","last_undegraded":"2024-09-21T15:34:13.052193+0000","last_fullsized":"2024-09-21T15:34:13.052193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":29180227,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":1961,"num_read_kb":8673,"num_write":252,"num_write_kb":8054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"82'166","reported_seq":404,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.065650+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:13.065650+0000","last_peered":"2024-09-21T15:34:13.065650+0000","last_clean":"2024-09-21T15:34:13.065650+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:13.065650+0000","last_undegraded":"2024-09-21T15:34:13.065650+0000","last_fullsized":"2024-09-21T15:34:13.065650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":30157686,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":291,"num_read_kb":3328,"num_write":187,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"78'316","reported_seq":17254,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.538412+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:13.538412+0000","last_peered":"2024-09-21T15:34:13.538412+0000","last_clean":"2024-09-21T15:34:13.538412+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:13.538412+0000","last_undegraded":"2024-09-21T15:34:13.538412+0000","last_fullsized":"2024-09-21T15:34:13.538412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14184493,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":17953,"num_read_kb":4691,"num_write":470,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"82'250","reported_seq":33277,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.192294+0000","last_change":"2024-09-21T15:33:29.818868+0000","last_active":"2024-09-21T15:34:13.192294+0000","last_peered":"2024-09-21T15:34:13.192294+0000","last_clean":"2024-09-21T15:34:13.192294+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:34:13.192294+0000","last_undegraded":"2024-09-21T15:34:13.192294+0000","last_fullsized":"2024-09-21T15:34:13.192294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:51:03.631593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":38989655,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":33697,"num_read_kb":32034,"num_write":287,"num_write_kb":20220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"82'265","reported_seq":1135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.193584+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:34:13.193584+0000","last_peered":"2024-09-21T15:34:13.193584+0000","last_clean":"2024-09-21T15:34:13.193584+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:34:13.193584+0000","last_undegraded":"2024-09-21T15:34:13.193584+0000","last_fullsized":"2024-09-21T15:34:13.193584+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":26799264,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":2070,"num_read_kb":4136,"num_write":308,"num_write_kb":2403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'185","reported_seq":494,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.540459+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:13.540459+0000","last_peered":"2024-09-21T15:34:13.540459+0000","last_clean":"2024-09-21T15:34:13.540459+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:13.540459+0000","last_undegraded":"2024-09-21T15:34:13.540459+0000","last_fullsized":"2024-09-21T15:34:13.540459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":12898891,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":546,"num_read_kb":5262,"num_write":178,"num_write_kb":4701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"78'350","reported_seq":4262,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.678624+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:13.678624+0000","last_peered":"2024-09-21T15:34:13.678624+0000","last_clean":"2024-09-21T15:34:13.678624+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:13.678624+0000","last_undegraded":"2024-09-21T15:34:13.678624+0000","last_fullsized":"2024-09-21T15:34:13.678624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":25206458,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":5646,"num_read_kb":17187,"num_write":483,"num_write_kb":17208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"81'292","reported_seq":3075,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.677243+0000","last_change":"2024-09-21T15:33:30.754648+0000","last_active":"2024-09-21T15:34:13.677243+0000","last_peered":"2024-09-21T15:34:13.677243+0000","last_clean":"2024-09-21T15:34:13.677243+0000","last_became_active":"2024-09-21T15:33:30.754007+0000","last_became_peered":"2024-09-21T15:33:30.754007+0000","last_unstale":"2024-09-21T15:34:13.677243+0000","last_undegraded":"2024-09-21T15:34:13.677243+0000","last_fullsized":"2024-09-21T15:34:13.677243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:51:12.343754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":10871771,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":3681,"num_read_kb":4979,"num_write":362,"num_write_kb":1948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":59879,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'258","reported_seq":1537,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T15:34:13.501841+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:13.501841+0000","last_peered":"2024-09-21T15:34:13.501841+0000","last_clean":"2024-09-21T15:34:13.501841+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:13.501841+0000","last_undegraded":"2024-09-21T15:34:13.501841+0000","last_fullsized":"2024-09-21T15:34:13.501841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":24086699,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":3546,"num_read_kb":4499,"num_write":251,"num_write_kb":10145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":680347546,"num_objects":1104,"num_object_clones":239,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1104,"num_whiteouts":451,"num_read":158967,"num_read_kb":254651,"num_write":11085,"num_write_kb":266640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":5406941,"num_keys_recovered":35,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138526720,"data_stored":163420898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8709,"ondisk_log_size":8709,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53068,"kb_used_data":26052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318772,"statfs":{"total":96636764160,"available":96582422528,"internally_reserved":0,"allocated":26677248,"data_stored":30099669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,12,0,16,35,107,68],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.309},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.626},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45108,"kb_used_data":9900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94326732,"statfs":{"total":96636764160,"available":96590573568,"internally_reserved":0,"allocated":10137600,"data_stored":10695440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.84799999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.1699999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0389999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.109},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.0660000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46944,"kb_used_data":19928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94324896,"statfs":{"total":96636764160,"available":96588693504,"internally_reserved":0,"allocated":20406272,"data_stored":22768810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.85999999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.046},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.60599999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.746},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.1100000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0089999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52504,"kb_used_data":25484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319336,"statfs":{"total":96636764160,"available":96583000064,"internally_reserved":0,"allocated":26095616,"data_stored":28559072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.437}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.497}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":2.3109999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65096,"kb_used_data":25792,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94306744,"statfs":{"total":96636764160,"available":96570105856,"internally_reserved":0,"allocated":26411008,"data_stored":31427654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.20300000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.22}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.318},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.26200000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.68000000000000005}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36432,"kb_used_data":5316,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335408,"statfs":{"total":96636764160,"available":96599457792,"internally_reserved":0,"allocated":5443584,"data_stored":5639116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.36299999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.44},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.47199999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.503},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.54500000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.79900000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.90500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48688,"kb_used_data":13476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94323152,"statfs":{"total":96636764160,"available":96586907648,"internally_reserved":0,"allocated":13799424,"data_stored":15544992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73899999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.89000000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.004},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.97899999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0369999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":17996,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":18427904,"data_stored":23134546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.56000000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.3120000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.3979999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.97099999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.464},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.095}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.0169999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16867328,"data_stored":22102060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12828672,"data_stored":15102855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4476928,"data_stored":5196998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25456640,"data_stored":30991708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25128960,"data_stored":28116954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19451904,"data_stored":22332880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8589312,"data_stored":9669126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25726976,"data_stored":29908317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:16.993 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [2, 6] 2024-09-21T15:34:16.993 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.14', '2', '6'] 2024-09-21T15:34:16.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.14 2 6 2024-09-21T15:34:16.993 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:16.984+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:17.088 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:17.084+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:17.149 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:17.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:17.149 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:17.149 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:17.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:17.196 INFO:teuthology.orchestra.run.smithi107.stdout:leaving image image_client.0-clone31 intact 2024-09-21T15:34:17.196 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:17.194+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:17.276 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:17.276 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:17.276 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:17.276 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:17.277 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:17.290 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:17.288+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:17.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:17.359 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T15:34:17.391 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:17.388+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:666 3.0 e26/21 3:029645da:::rbd_data.11911fd826fd.0000000000000040:head v 26'9, mlcod=26'7)", 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.918345708999993, 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00333943, 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543942+0000", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543939+0000", 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:17.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.543962+0000", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3096e-05 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544008+0000", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6282999999999997e-05 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544029+0000", 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0673999999999999e-05 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544168+0000", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000139472 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.544266+0000", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7461999999999996e-05 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547194+0000", 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0029280840000000001 2024-09-21T15:34:17.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.547281+0000", 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7078999999999998e-05 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3288 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'18, mlcod=29'16)", 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.447429438, 2024-09-21T15:34:17.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034309169999999999, 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014858+0000", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014856+0000", 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014868+0000", 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2069000000000001e-05 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014885+0000", 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6884e-05 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.469 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014896+0000", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0307e-05 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014957+0000", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0985e-05 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.014987+0000", 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0352e-05 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018255+0000", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032679649999999998 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.018289+0000", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.383e-05 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.437639345999997, 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:34:17.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.423685305999996, 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:34:17.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:34:17.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.479 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3380 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'21, mlcod=30'19)", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.415518160999994, 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00352951, 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046769+0000", 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046768+0000", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046777+0000", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3510000000000008e-06 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046789+0000", 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2157999999999999e-05 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046796+0000", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6209999999999998e-06 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046847+0000", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1022e-05 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.046872+0000", 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4932000000000001e-05 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050255+0000", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033831769999999998 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.050299+0000", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3361e-05 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:17.483 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.414771762000001, 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:17.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3451 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'24, mlcod=30'22)", 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.396380785999995, 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034045339999999999, 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065906+0000", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065905+0000", 2024-09-21T15:34:17.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065912+0000", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3769999999999998e-06 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065924+0000", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1464999999999999e-05 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065930+0000", 2024-09-21T15:34:17.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6879999999999996e-06 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065970+0000", 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0908000000000002e-05 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.065985+0000", 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4168000000000001e-05 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069278+0000", 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032936990000000002 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.069311+0000", 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.269e-05 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3481 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'30, mlcod=30'28)", 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.335514845999995, 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034998569999999999, 2024-09-21T15:34:17.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126772+0000", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126770+0000", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126803+0000", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2805999999999997e-05 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126832+0000", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8983e-05 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126847+0000", 2024-09-21T15:34:17.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.465e-05 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126889+0000", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2209000000000002e-05 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.126908+0000", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8842999999999999e-05 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130220+0000", 2024-09-21T15:34:17.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033122989999999999 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.130272+0000", 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2234000000000002e-05 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.323844510000001, 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:17.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7321 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'202, mlcod=36'199)", 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.365369412000007, 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035863610000000001, 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.499 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096918+0000", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096917+0000", 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:17.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096922+0000", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7350000000000004e-06 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096931+0000", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1090000000000004e-06 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096936+0000", 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4839999999999998e-06 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096976+0000", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0642000000000002e-05 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.096990+0000", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3278999999999999e-05 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100480+0000", 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034907559999999998 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.100504+0000", 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3917000000000002e-05 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.364458556000002, 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.363492629000007, 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.353288646999999, 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:17.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:17.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:17.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:15192 3.0 e58/50 3:002446f9:::rbd_data.11919eb7aeb1.000000000000004f:head v 58'302, mlcod=58'298)", 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.573574299999997, 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032278039999999999, 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888713+0000", 2024-09-21T15:34:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888712+0000", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888728+0000", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5973000000000001e-05 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888739+0000", 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1039999999999999e-05 2024-09-21T15:34:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888745+0000", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6049999999999997e-06 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888780+0000", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5373999999999997e-05 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.888800+0000", 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9565e-05 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891899+0000", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030989469999999999 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:30.891941+0000", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2187000000000001e-05 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62123 3.4 e68/47 3:22ecafc8:::rbd_data.1191e6bca1cd.00000000000000a8:head v 68'99, mlcod=65'97)", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.869957277000001, 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032546599999999999, 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592330+0000", 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592327+0000", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592348+0000", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1233000000000002e-05 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592397+0000", 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8353000000000003e-05 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592417+0000", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0653e-05 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592769+0000", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00035143300000000001 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.592822+0000", 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3137000000000003e-05 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595483+0000", 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026611849999999999 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.595585+0000", 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101462 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62269 3.0 e68/50 3:072ab387:::rbd_header.1191e6bca1cd:head v 68'364, mlcod=68'362)", 2024-09-21T15:34:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.583746950999998, 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037514879999999999, 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878538+0000", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878557+0000", 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9173000000000001e-05 2024-09-21T15:34:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878592+0000", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5429000000000002e-05 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878610+0000", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7974000000000001e-05 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878698+0000", 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7293000000000006e-05 2024-09-21T15:34:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878730+0000", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2175000000000003e-05 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882213+0000", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034834689999999999 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882292+0000", 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8453000000000003e-05 2024-09-21T15:34:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62282 3.0 e69/50 3:01c4b5a3:::rbd_data.1191e6bca1cd.00000000000000da:head v 69'365, mlcod=68'363)", 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.559621663000001, 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037099500000000001, 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902665+0000", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902697+0000", 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1680000000000002e-05 2024-09-21T15:34:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902716+0000", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9145999999999999e-05 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902726+0000", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0139000000000001e-05 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902782+0000", 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5986999999999998e-05 2024-09-21T15:34:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902801+0000", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8300000000000001e-05 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906328+0000", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035275829999999999 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906376+0000", 2024-09-21T15:34:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:102949 3.4 e72/47 3:221b15ed:::rbd_data.1191b30e35a1.0000000000000042:head v 72'106, mlcod=72'104)", 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 20.125708328999998, 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040397410000000003, 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336577+0000", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336598+0000", 2024-09-21T15:34:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1659e-05 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336622+0000", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3204e-05 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336633+0000", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0967e-05 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336686+0000", 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3915999999999997e-05 2024-09-21T15:34:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336712+0000", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5060999999999999e-05 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340579+0000", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038670309999999999 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340619+0000", 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0080999999999998e-05 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:103104 3.9 e72/47 3:92ad05bc:::rbd_data.1191b30e35a1.000000000000002d:head v 72'107, mlcod=72'105)", 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.984197964, 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034052539999999999, 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478089+0000", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478088+0000", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478140+0000", 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2240000000000001e-05 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478160+0000", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0176000000000001e-05 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478172+0000", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2133999999999999e-05 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478283+0000", 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110418 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.478348+0000", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4735999999999998e-05 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481393+0000", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030456789999999999 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.481495+0000", 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101309 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.9377273499999994, 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:17.545 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:17.546 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:17.546 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:17.542+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:17.647 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:17.642+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:17.662 INFO:teuthology.orchestra.run.smithi107.stdout:4221 trunc from 0xb59e808 to 0x9eb8cba 2024-09-21T15:34:17.678 INFO:teuthology.orchestra.run.smithi107.stdout:4222 punch from 0x2fed44e to 0x2ff4462, (0x7014 bytes) 2024-09-21T15:34:17.678 INFO:teuthology.orchestra.run.smithi107.stdout:4224 write 0x4b48a3b thru 0x4b4d2ea (0x48b0 bytes) 2024-09-21T15:34:17.683 INFO:teuthology.orchestra.run.smithi107.stdout:4225 write 0x6f3bad9 thru 0x6f3f61a (0x3b42 bytes) 2024-09-21T15:34:17.690 INFO:teuthology.orchestra.run.smithi107.stdout:4227 trunc from 0x9eb8cba to 0xc8d78f2 2024-09-21T15:34:17.703 INFO:teuthology.orchestra.run.smithi107.stdout:4231 writesame 0xdc8580d thru 0xdc943ad data_size 0x1(0xeba1 bytes) 2024-09-21T15:34:17.703 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.703 INFO:teuthology.orchestra.run.smithi107.stdout:4233 write 0xcd6b654 thru 0xcd716ac (0x6059 bytes) 2024-09-21T15:34:17.705 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:17.705 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:17.705 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:17.705 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:17.707 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.707 INFO:teuthology.orchestra.run.smithi107.stdout:4238 write 0x8555dff thru 0x856087f (0xaa81 bytes) 2024-09-21T15:34:17.713 INFO:teuthology.orchestra.run.smithi107.stdout:4240 read 0x3750fa1 thru 0x375cd15 (0xbd75 bytes) 2024-09-21T15:34:17.713 INFO:teuthology.orchestra.run.smithi107.stdout:4241 write 0x93b7418 thru 0x93c4444 (0xd02d bytes) 2024-09-21T15:34:17.719 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:17.719 INFO:teuthology.orchestra.run.smithi107.stdout:4242 write 0x98a90a9 thru 0x98b0232 (0x718a bytes) 2024-09-21T15:34:17.736 INFO:teuthology.orchestra.run.smithi107.stdout:4243 trunc from 0xdc943ae to 0x3d57e76 2024-09-21T15:34:17.748 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:17.744+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:17.763 INFO:teuthology.orchestra.run.smithi107.stdout:4245 read 0x2c02d4a thru 0x2c0650d (0x37c4 bytes) 2024-09-21T15:34:17.763 INFO:teuthology.orchestra.run.smithi107.stdout:4246 read 0x31949ca thru 0x31a2017 (0xd64e bytes) 2024-09-21T15:34:17.764 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.764 INFO:teuthology.orchestra.run.smithi107.stdout:4249 read 0x23d5554 thru 0x23e394c (0xe3f9 bytes) 2024-09-21T15:34:17.764 INFO:teuthology.orchestra.run.smithi107.stdout:4250 read 0x38150a3 thru 0x38241d4 (0xf132 bytes) 2024-09-21T15:34:17.764 INFO:teuthology.orchestra.run.smithi107.stdout:4253 write 0xb3f081 thru 0xb47802 (0x8782 bytes) 2024-09-21T15:34:17.779 INFO:teuthology.orchestra.run.smithi107.stdout:4254 read 0x3a14420 thru 0x3a14b77 (0x758 bytes) 2024-09-21T15:34:17.783 INFO:teuthology.orchestra.run.smithi107.stdout:4256 trunc from 0x3d57e76 to 0x6ab08f8 2024-09-21T15:34:17.789 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.798 INFO:teuthology.orchestra.run.smithi107.stdout:4258 writesame 0x80653c5 thru 0x806cf0e data_size 0x2b(0x7b4a bytes) 2024-09-21T15:34:17.799 INFO:teuthology.orchestra.run.smithi107.stdout:4261 read 0x7688fc3 thru 0x7698a7c (0xfaba bytes) 2024-09-21T15:34:17.799 INFO:teuthology.orchestra.run.smithi107.stdout:4262 write 0x13369b0 thru 0x133fea9 (0x94fa bytes) 2024-09-21T15:34:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:4263 write 0x5311376 thru 0x531db43 (0xc7ce bytes) 2024-09-21T15:34:17.807 INFO:teuthology.orchestra.run.smithi107.stdout:4264 read 0x462f41d thru 0x463a88d (0xb471 bytes) 2024-09-21T15:34:17.807 INFO:teuthology.orchestra.run.smithi107.stdout:4265 read 0x7899733 thru 0x78a4145 (0xaa13 bytes) 2024-09-21T15:34:17.807 INFO:teuthology.orchestra.run.smithi107.stdout:4267 read 0x3766525 thru 0x376cdf6 (0x68d2 bytes) 2024-09-21T15:34:17.821 INFO:teuthology.orchestra.run.smithi107.stdout:4268 write 0xc8b4d64 thru 0xc8b7969 (0x2c06 bytes) 2024-09-21T15:34:17.827 INFO:teuthology.orchestra.run.smithi107.stdout:4271 write 0x77c897b thru 0x77d5759 (0xcddf bytes) 2024-09-21T15:34:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:4272 writesame 0xa5f594 thru 0xa632ae data_size 0x1(0x3d1b bytes) 2024-09-21T15:34:17.849 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:17.848+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:17.857 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:17.857 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:17.857 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:17.858 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:17.858 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:17.859 INFO:teuthology.orchestra.run.smithi107.stdout:4273 write 0x66cc2a6 thru 0x66d404c (0x7da7 bytes) 2024-09-21T15:34:17.866 INFO:teuthology.orchestra.run.smithi107.stdout:4275 punch from 0x79de761 to 0x79e278c, (0x402b bytes) 2024-09-21T15:34:17.866 INFO:teuthology.orchestra.run.smithi107.stdout:4276 writesame 0xa2378f0 thru 0xa23e356 data_size 0x1(0x6a67 bytes) 2024-09-21T15:34:17.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:17.882 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.882 INFO:teuthology.orchestra.run.smithi107.stdout:4278 read 0x78c87e9 thru 0x78ce1a6 (0x59be bytes) 2024-09-21T15:34:17.882 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.898 INFO:teuthology.orchestra.run.smithi107.stdout:4281 trunc from 0xc8b796a to 0x4b96849 2024-09-21T15:34:17.936 INFO:teuthology.orchestra.run.smithi107.stdout:4282 write 0xb143829 thru 0xb151513 (0xdceb bytes) 2024-09-21T15:34:17.939 INFO:teuthology.orchestra.run.smithi107.stdout:4284 write 0x13bcb28 thru 0x13cb1d1 (0xe6aa bytes) 2024-09-21T15:34:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:4285 trunc from 0xb151514 to 0xd7ce962 2024-09-21T15:34:17.950 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:17.946+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:4286 writesame 0x812d2d0 thru 0x812fa1a data_size 0x7(0x274b bytes) 2024-09-21T15:34:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:4287 writesame 0xdaecaee thru 0xdaf7d98 data_size 0x1(0xb2ab bytes) 2024-09-21T15:34:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:4288 punch from 0x521d859 to 0x522192e, (0x40d5 bytes) 2024-09-21T15:34:17.963 INFO:teuthology.orchestra.run.smithi107.stdout:4289 write 0x716a175 thru 0x716acb8 (0xb44 bytes) 2024-09-21T15:34:17.968 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:17.968 INFO:teuthology.orchestra.run.smithi107.stdout:4291 writesame 0x376501c thru 0x376ae19 data_size 0x2(0x5dfe bytes) 2024-09-21T15:34:17.984 INFO:teuthology.orchestra.run.smithi107.stdout:4292 write 0x62c5484 thru 0x62cf743 (0xa2c0 bytes) 2024-09-21T15:34:18.003 INFO:teuthology.orchestra.run.smithi107.stdout:4293 trunc from 0xdaf7d99 to 0x1584aec 2024-09-21T15:34:18.014 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:18.014 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:18.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.643529950000001, 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:18.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:18.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:18.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.020 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.615318774000002, 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:18.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:18.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.426019405999995, 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.294634715000001, 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:18.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:18.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.133537310999998, 2024-09-21T15:34:18.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:18.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.133484764000002, 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.133436638999996, 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:18.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:18.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.121039750999998, 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:18.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:18.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:18.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:18.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:18.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.560266768999995, 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:18.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:18.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.061 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.493242402999996, 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.062 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:18.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.358122272000003, 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.067 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:18.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.317460128999997, 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:18.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:18.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:18.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.317446911999994, 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:18.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.317434105000004, 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.317393698000004, 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.089 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.317369229999997, 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.298147241999999, 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:18.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:18.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:18.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.210843659000005, 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:18.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:18.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:18.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 11.216158216, 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:18.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.801898869, 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:18.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:18.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:18.113 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:18.113 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:18.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:34:18.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4294 trunc from 0x1584aec to 0x3aaf4fe 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4295 writesame 0x9616183 thru 0x961d66d data_size 0xb(0x74eb bytes) 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4297 write 0x86b4c94 thru 0x86b5a78 (0xde5 bytes) 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4298 write 0x5d238d7 thru 0x5d3314a (0xf874 bytes) 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4301 punch from 0x307d09a to 0x30848b8, (0x781e bytes) 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.115 INFO:teuthology.orchestra.run.smithi107.stdout:4304 writesame 0x596ed7f thru 0x596f167 data_size 0xb(0x3e9 bytes) 2024-09-21T15:34:18.116 INFO:teuthology.orchestra.run.smithi107.stderr:set 3.14 pg_upmap mapping to [2,6] 2024-09-21T15:34:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:4306 read 0x60973e3 thru 0x60a2e3c (0xba5a bytes) 2024-09-21T15:34:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:4307 writesame 0x90b75e4 thru 0x90bc224 data_size 0x1b(0x4c41 bytes) 2024-09-21T15:34:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.117 INFO:teuthology.orchestra.run.smithi107.stdout:4309 writesame 0x867a2f5 thru 0x8686ced data_size 0x2d(0xc9f9 bytes) 2024-09-21T15:34:18.117 INFO:teuthology.orchestra.run.smithi107.stdout:4310 read 0x9163771 thru 0x9167f48 (0x47d8 bytes) 2024-09-21T15:34:18.117 INFO:teuthology.orchestra.run.smithi107.stdout:4311 trunc from 0x961d66e to 0x6a29ec1 2024-09-21T15:34:18.117 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:18.112+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:18.124 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.131 INFO:teuthology.orchestra.run.smithi107.stdout:4314 write 0x73746c4 thru 0x737cb5f (0x849c bytes) 2024-09-21T15:34:18.135 INFO:teuthology.orchestra.run.smithi107.stdout:4316 write 0x69f94ac thru 0x6a030d9 (0x9c2e bytes) 2024-09-21T15:34:18.140 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.140 INFO:teuthology.orchestra.run.smithi107.stdout:4318 write 0x7349328 thru 0x734aef4 (0x1bcd bytes) 2024-09-21T15:34:18.142 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.142 INFO:teuthology.orchestra.run.smithi107.stdout:4321 read 0x38bfcf6 thru 0x38c935b (0x9666 bytes) 2024-09-21T15:34:18.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:18.154 INFO:teuthology.orchestra.run.smithi107.stdout:4322 writesame 0xbad99a9 thru 0xbae25d3 data_size 0x1b(0x8c2b bytes) 2024-09-21T15:34:18.154 INFO:teuthology.orchestra.run.smithi107.stdout:4323 read 0x8478b73 thru 0x848072c (0x7bba bytes) 2024-09-21T15:34:18.154 INFO:teuthology.orchestra.run.smithi107.stdout:4324 read 0x6b2ee11 thru 0x6b33d25 (0x4f15 bytes) 2024-09-21T15:34:18.155 INFO:teuthology.orchestra.run.smithi107.stdout:4326 read 0x72b8744 thru 0x72ba920 (0x21dd bytes) 2024-09-21T15:34:18.155 INFO:teuthology.orchestra.run.smithi107.stdout:4327 punch from 0x1483da3 to 0x1489f2f, (0x618c bytes) 2024-09-21T15:34:18.155 INFO:teuthology.orchestra.run.smithi107.stdout:4328 punch from 0x6195650 to 0x61a11bc, (0xbb6c bytes) 2024-09-21T15:34:18.155 INFO:teuthology.orchestra.run.smithi107.stdout:4329 write 0x31b96a4 thru 0x31c8800 (0xf15d bytes) 2024-09-21T15:34:18.160 INFO:teuthology.orchestra.run.smithi107.stdout:4330 read 0x31c6301 thru 0x31c6ecf (0xbcf bytes) 2024-09-21T15:34:18.163 INFO:teuthology.orchestra.run.smithi107.stdout:4331 writesame 0x834ea50 thru 0x835a6d6 data_size 0x11(0xbc87 bytes) 2024-09-21T15:34:18.168 INFO:teuthology.orchestra.run.smithi107.stdout:4341 writesame 0x7b90687 thru 0x7b9f536 data_size 0x98(0xeeb0 bytes) 2024-09-21T15:34:18.185 INFO:teuthology.orchestra.run.smithi107.stdout:4342 trunc from 0xbae25d4 to 0x8b3cfc 2024-09-21T15:34:18.203 INFO:teuthology.orchestra.run.smithi107.stdout:4343 read 0x7f77a6 thru 0x7f90b7 (0x1912 bytes) 2024-09-21T15:34:18.204 INFO:teuthology.orchestra.run.smithi107.stdout:4345 trunc from 0x8b3cfc to 0xd2c47d 2024-09-21T15:34:18.215 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:18.212+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:18.235 INFO:teuthology.orchestra.run.smithi107.stdout:4346 writesame 0xedace48 thru 0xedb3eee data_size 0x1(0x70a7 bytes) 2024-09-21T15:34:18.236 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.236 INFO:teuthology.orchestra.run.smithi107.stdout:4349 read 0x99c670a thru 0x99c81ac (0x1aa3 bytes) 2024-09-21T15:34:18.236 INFO:teuthology.orchestra.run.smithi107.stdout:4350 write 0x40a3a70 thru 0x40aa737 (0x6cc8 bytes) 2024-09-21T15:34:18.240 INFO:teuthology.orchestra.run.smithi107.stdout:4351 write 0x23c3d13 thru 0x23cfdc2 (0xc0b0 bytes) 2024-09-21T15:34:18.246 INFO:teuthology.orchestra.run.smithi107.stdout:4352 writesame 0xe1e701b thru 0xe1f0f19 data_size 0x1f(0x9eff bytes) 2024-09-21T15:34:18.252 INFO:teuthology.orchestra.run.smithi107.stdout:4353 read 0xb6a87fb thru 0xb6ae899 (0x609f bytes) 2024-09-21T15:34:18.252 INFO:teuthology.orchestra.run.smithi107.stdout:4356 write 0x6df2403 thru 0x6df7b7b (0x5779 bytes) 2024-09-21T15:34:18.253 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:18.253 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:18.253 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:18.253 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:18.262 INFO:teuthology.orchestra.run.smithi107.stdout:4357 trunc from 0xedb3eef to 0xaccb7a1 2024-09-21T15:34:18.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:18.278 INFO:teuthology.orchestra.run.smithi107.stdout:4358 trunc from 0xaccb7a1 to 0xeb22967 2024-09-21T15:34:18.284 INFO:teuthology.orchestra.run.smithi107.stdout:4359 write 0x1bccc02 thru 0x1bce56d (0x196c bytes) 2024-09-21T15:34:18.288 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.293 INFO:teuthology.orchestra.run.smithi107.stdout:4363 write 0xecffe29 thru 0xed0e975 (0xeb4d bytes) 2024-09-21T15:34:18.303 INFO:teuthology.orchestra.run.smithi107.stdout:4366 trunc from 0xed0e976 to 0xad01a76 2024-09-21T15:34:18.304 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:18.304 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:18.315 INFO:teuthology.orchestra.run.smithi107.stdout:4368 read 0x838d975 thru 0x8397aa8 (0xa134 bytes) 2024-09-21T15:34:18.315 INFO:teuthology.orchestra.run.smithi107.stdout:4369 write 0x7d44ad2 thru 0x7d48c5c (0x418b bytes) 2024-09-21T15:34:18.316 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:18.314+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:18.316 INFO:teuthology.orchestra.run.smithi107.stdout:4371 punch from 0x308c78c to 0x3098ccf, (0xc543 bytes) 2024-09-21T15:34:18.316 INFO:teuthology.orchestra.run.smithi107.stdout:4372 read 0x5ac3510 thru 0x5acb98d (0x847e bytes) 2024-09-21T15:34:18.316 INFO:teuthology.orchestra.run.smithi107.stdout:4373 read 0x3775125 thru 0x37799a5 (0x4881 bytes) 2024-09-21T15:34:18.317 INFO:teuthology.orchestra.run.smithi107.stdout:4376 write 0x6a79fe5 thru 0x6a7d1f5 (0x3211 bytes) 2024-09-21T15:34:18.320 INFO:teuthology.orchestra.run.smithi107.stdout:4377 punch from 0x2bfabba to 0x2c063ab, (0xb7f1 bytes) 2024-09-21T15:34:18.320 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.320 INFO:teuthology.orchestra.run.smithi107.stdout:4379 read 0x5069ad6 thru 0x506f66a (0x5b95 bytes) 2024-09-21T15:34:18.320 INFO:teuthology.orchestra.run.smithi107.stdout:4380 write 0x938e84c thru 0x939e1cf (0xf984 bytes) 2024-09-21T15:34:18.323 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:18.323 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:18.333 INFO:teuthology.orchestra.run.smithi107.stdout:4382 trunc from 0xad01a76 to 0x3b2df01 2024-09-21T15:34:18.341 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:18.341 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:18.347 INFO:teuthology.orchestra.run.smithi107.stdout:4384 read 0x1cf5ce0 thru 0x1cfbd2c (0x604d bytes) 2024-09-21T15:34:18.358 INFO:teuthology.orchestra.run.smithi107.stdout:4385 trunc from 0x3b2df01 to 0xcd2d2ba 2024-09-21T15:34:18.360 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:18.360 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:18.364 INFO:teuthology.orchestra.run.smithi107.stdout:4386 write 0xa4ded3e thru 0xa4ed300 (0xe5c3 bytes) 2024-09-21T15:34:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:4389 read 0xa31be75 thru 0xa3241c0 (0x834c bytes) 2024-09-21T15:34:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:4390 punch from 0x5d9078b to 0x5d9726a, (0x6adf bytes) 2024-09-21T15:34:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:4391 read 0xd8e0d7 thru 0xd8ff03 (0x1e2d bytes) 2024-09-21T15:34:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:4392 read 0x16b4ba6 thru 0x16c3bc6 (0xf021 bytes) 2024-09-21T15:34:18.370 INFO:teuthology.orchestra.run.smithi107.stdout:4393 trunc from 0xcd2d2ba to 0xcb7f3f4 2024-09-21T15:34:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:4394 punch from 0x94516d6 to 0x945daa4, (0xc3ce bytes) 2024-09-21T15:34:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:4395 read 0xbbd46ca thru 0xbbd84c1 (0x3df8 bytes) 2024-09-21T15:34:18.380 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:18.381 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:18.388 INFO:teuthology.orchestra.run.smithi107.stdout:4397 trunc from 0xcb7f3f4 to 0x25fefca 2024-09-21T15:34:18.402 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:18.408 INFO:teuthology.orchestra.run.smithi107.stdout:4399 writesame 0x49d662f thru 0x49dd829 data_size 0x1(0x71fb bytes) 2024-09-21T15:34:18.408 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.409 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.409 INFO:teuthology.orchestra.run.smithi107.stdout:4402 read 0x1406ce1 thru 0x140c853 (0x5b73 bytes) 2024-09-21T15:34:18.409 INFO:teuthology.orchestra.run.smithi107.stdout:4403 punch from 0x37bcf6c to 0x37ccdeb, (0xfe7f bytes) 2024-09-21T15:34:18.416 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:18.412+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:18.420 INFO:teuthology.orchestra.run.smithi107.stdout:4404 writesame 0x9ce7b06 thru 0x9cf2f7c data_size 0x1(0xb477 bytes) 2024-09-21T15:34:18.422 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:18.422 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:18.425 INFO:teuthology.orchestra.run.smithi107.stdout:4405 trunc from 0x9cf2f7d to 0x598873a 2024-09-21T15:34:18.438 INFO:teuthology.orchestra.run.smithi107.stdout:4407 writesame 0x620818c thru 0x6216907 data_size 0xa(0xe77c bytes) 2024-09-21T15:34:18.439 INFO:teuthology.orchestra.run.smithi107.stdout:4408 read 0x476159b thru 0x476338d (0x1df3 bytes) 2024-09-21T15:34:18.439 INFO:teuthology.orchestra.run.smithi107.stdout:4411 read 0x50b171b thru 0x50bb6a3 (0x9f89 bytes) 2024-09-21T15:34:18.439 INFO:teuthology.orchestra.run.smithi107.stdout:4412 writesame 0x3c9f854 thru 0x3ca95e9 data_size 0x17(0x9d96 bytes) 2024-09-21T15:34:18.440 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:18.440 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:18.446 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:18.446 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:18.453 INFO:teuthology.orchestra.run.smithi107.stdout:4413 writesame 0x99f6e16 thru 0x99f91f3 data_size 0x1(0x23de bytes) 2024-09-21T15:34:18.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:18.463 INFO:teuthology.orchestra.run.smithi107.stdout:4414 write 0xc5043b9 thru 0xc51371f (0xf367 bytes) 2024-09-21T15:34:18.467 INFO:teuthology.orchestra.run.smithi107.stdout:4417 read 0xbb597ca thru 0xbb67e24 (0xe65b bytes) 2024-09-21T15:34:18.467 INFO:teuthology.orchestra.run.smithi107.stdout:4419 write 0xabdd8ab thru 0xabe8af7 (0xb24d bytes) 2024-09-21T15:34:18.470 INFO:teuthology.orchestra.run.smithi107.stdout:4420 write 0x89606b5 thru 0x896fcd7 (0xf623 bytes) 2024-09-21T15:34:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:4421 read 0xb33bfac thru 0xb343418 (0x746d bytes) 2024-09-21T15:34:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:4422 read 0x891e890 thru 0x8925d82 (0x74f3 bytes) 2024-09-21T15:34:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:4423 read 0x9b772e1 thru 0x9b803b3 (0x90d3 bytes) 2024-09-21T15:34:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:4424 write 0x4746559 thru 0x474aea8 (0x4950 bytes) 2024-09-21T15:34:18.478 INFO:teuthology.orchestra.run.smithi107.stdout:4425 trunc from 0xc513720 to 0xd6fa2e6 2024-09-21T15:34:18.483 INFO:teuthology.orchestra.run.smithi107.stdout:4426 write 0xa09b964 thru 0xa0a98ea (0xdf87 bytes) 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:4428 read 0x91b42ed thru 0x91bc379 (0x808d bytes) 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:4430 read 0xe9c7b2 thru 0xeaae15 (0xe664 bytes) 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:4431 punch from 0x4b60763 to 0x4b6b6f5, (0xaf92 bytes) 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:18.487 INFO:teuthology.orchestra.run.smithi107.stdout:4433 clone 33 order 18 su 32768 sc 11 2024-09-21T15:34:18.517 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:18.514+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:18.618 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:18.618 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:18.618 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:18.618+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:18.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:18.719 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:18.716+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:18.756 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:18.756 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:18.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:18.820 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:18.818+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:18.920 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:18.916+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:18.936 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:18.937 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:18.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:19.021 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:19.018+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:19.101 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:34:19.121 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:19.120+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:19.121 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:19.121 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:19.136 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:19.198 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone32 from 0xd6fa2e6 (overlap 0x8b3cfc) to 0x37b1fe 2024-09-21T15:34:19.222 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:19.218+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:19.307 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:19.308 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:19.322 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:19.322 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:19.320+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T15:34:19.423 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:19.420+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:19.503 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:19.503 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:19.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:19.561 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:19.560+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:19.625 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:19.624+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:19.683 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:19.684 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:19.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:19.741 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:19.740+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:19.826 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:19.824+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:19.864 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:19.864 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:19.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:19.965 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:19.964+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:20.028 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:20.026+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:20.038 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:20.038 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:20.052 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:20.141 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:20.140+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:20.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:20.269 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:20.268+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:20.330 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:20.328+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:20.357 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:20.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:20.431 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:20.428+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:20.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:20.485 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:20.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:20.548 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:20.546+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:20.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:20.611 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:20.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:20.628 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:20.632 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:20.630+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:20.646 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:20.646 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:20.665 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:20.665 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:20.673 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:20.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4210 3.1 3:81178acd:::rbd_data.119117b26f37.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.962661179999998, 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049998918000000003, 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4210 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709667+0000", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.709665+0000", 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717794+0000", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0081293570000000002 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717822+0000", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7923999999999999e-05 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717836+0000", 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3732e-05 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717917+0000", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0778999999999993e-05 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718051+0000", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000134234 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 4", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718471+0000", 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00042049299999999997 2024-09-21T15:34:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.749616+0000", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.031145029000000001 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759555+0000", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0099382250000000002 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759617+0000", 2024-09-21T15:34:20.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2176999999999998e-05 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.759666+0000", 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9517e-05 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:5402 3.12 3:4b7f43d8:::rbd_data.11917ba4f9b.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.024841660000007, 2024-09-21T15:34:20.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.058211671, 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5402 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647487+0000", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647486+0000", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647492+0000", 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9229999999999999e-06 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647506+0000", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4076999999999999e-05 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647513+0000", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0530000000000001e-06 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647627+0000", 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000113755 2024-09-21T15:34:20.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.647846+0000", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021904800000000001 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.659745+0000", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011899138 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705558+0000", 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.045812840000000001 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705608+0000", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9838e-05 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705658+0000", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0213000000000002e-05 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:01.705699+0000", 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0605000000000002e-05 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.684 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124484 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2975744~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.30121215, 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043060238000000001, 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124484 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371117+0000", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371116+0000", 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371118+0000", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9460000000000001e-06 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371123+0000", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3700000000000003e-06 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371127+0000", 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0280000000000002e-06 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414044+0000", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042916081000000002 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414115+0000", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1607999999999996e-05 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414177+0000", 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1617000000000003e-05 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.688 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124485 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2985984~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.30119762, 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043229165, 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124485 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.689 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371131+0000", 2024-09-21T15:34:20.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371133+0000", 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8899999999999999e-06 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371138+0000", 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1359999999999996e-06 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371147+0000", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.8340000000000005e-06 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414210+0000", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043063203000000001 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414291+0000", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1428999999999996e-05 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414360+0000", 2024-09-21T15:34:20.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9033000000000001e-05 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124486 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 2996224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301178520000001, 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043374530000000001, 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124486 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371150+0000", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371152+0000", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.888e-06 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371157+0000", 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2560000000000002e-06 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371161+0000", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0899999999999998e-06 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414392+0000", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043231251999999998 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414461+0000", 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8369999999999998e-05 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414525+0000", 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4046000000000003e-05 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124487 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3006464~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301164102, 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043541654999999999, 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124487 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371165+0000", 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371164+0000", 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371166+0000", 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7460000000000001e-06 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371171+0000", 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0719999999999999e-06 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371175+0000", 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0659999999999997e-06 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414556+0000", 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043381182999999997 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414636+0000", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9809999999999997e-05 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414706+0000", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0134000000000003e-05 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124488 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3016704~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:20.701 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301150238, 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692364999999997, 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124488 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371178+0000", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371180+0000", 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.734e-06 2024-09-21T15:34:20.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371185+0000", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371189+0000", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.912e-06 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414738+0000", 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043548877 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414809+0000", 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1675999999999996e-05 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414871+0000", 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1477999999999997e-05 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124489 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3026944~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.705 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301136302, 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043866982999999998, 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124489 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:20.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371192+0000", 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371194+0000", 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371199+0000", 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0220000000000004e-06 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371203+0000", 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.941e-06 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414912+0000", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043709439000000003 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.414992+0000", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9572000000000003e-05 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415059+0000", 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7539000000000005e-05 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.709 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124490 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3037184~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301122421000001, 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044018785999999997, 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124490 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371206+0000", 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371208+0000", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8080000000000001e-06 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371213+0000", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0440000000000003e-06 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371217+0000", 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0300000000000004e-06 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415092+0000", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043875441000000001 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415163+0000", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1026999999999996e-05 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415225+0000", 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1793999999999998e-05 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124491 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3047424~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301108357, 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044189697999999999, 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124491 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371220+0000", 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371230+0000", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0068000000000001e-05 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371244+0000", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.395e-05 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371251+0000", 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1960000000000002e-06 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415258+0000", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044007019000000001 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415339+0000", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1036000000000001e-05 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415410+0000", 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0784999999999994e-05 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124492 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3057664~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.301052937, 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044296730999999999, 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124492 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371276+0000", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371275+0000", 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371279+0000", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5260000000000002e-06 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371291+0000", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2098e-05 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.371307+0000", 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6787999999999999e-05 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415440+0000", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044132678000000002 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415510+0000", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9999999999999994e-05 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415572+0000", 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2333000000000001e-05 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124495 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3088384~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299362606000001, 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043138292000000002, 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124495 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372966+0000", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372965+0000", 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372969+0000", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0119999999999999e-06 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372982+0000", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2517000000000001e-05 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.372991+0000", 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4180000000000003e-06 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.415952+0000", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042960920999999999 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416034+0000", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1464999999999993e-05 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416104+0000", 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0635000000000004e-05 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124496 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3098624~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299327043, 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043270034999999998, 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124496 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373002+0000", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373001+0000", 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373005+0000", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7340000000000002e-06 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373020+0000", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5622000000000001e-05 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373029+0000", 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0240000000000003e-06 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416137+0000", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043107686999999999 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416206+0000", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999995e-05 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416272+0000", 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5283999999999994e-05 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124497 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3108864~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299290093, 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043416800999999998, 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124497 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373039+0000", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373038+0000", 2024-09-21T15:34:20.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373041+0000", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5880000000000002e-06 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373052+0000", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.057e-05 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373060+0000", 2024-09-21T15:34:20.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8720000000000002e-06 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416303+0000", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043243216000000001 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416386+0000", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2712000000000004e-05 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416455+0000", 2024-09-21T15:34:20.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9535000000000004e-05 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124498 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3119104~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299259787, 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043553477, 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.734 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124498 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373069+0000", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373068+0000", 2024-09-21T15:34:20.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373072+0000", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3220000000000001e-06 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373082+0000", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0368e-05 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373100+0000", 2024-09-21T15:34:20.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8309e-05 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416488+0000", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043387362999999998 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416560+0000", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2568e-05 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416622+0000", 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2212999999999995e-05 2024-09-21T15:34:20.737 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124499 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3129344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299220997999999, 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043692511000000003, 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124499 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373108+0000", 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373107+0000", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373111+0000", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373122+0000", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1590000000000001e-05 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373131+0000", 2024-09-21T15:34:20.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3519999999999996e-06 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416653+0000", 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043521988999999997 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416733+0000", 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9479000000000003e-05 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416800+0000", 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7218999999999995e-05 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124500 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3139584~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.299179248, 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043824758999999998, 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124500 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373149+0000", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373152+0000", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.58e-06 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373166+0000", 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3835999999999999e-05 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.373176+0000", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5019999999999998e-06 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416833+0000", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043656815000000002 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416911+0000", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8788999999999994e-05 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.416974+0000", 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2854999999999994e-05 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124521 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4077568~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.294705927000001, 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043139638000000001, 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.746 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124521 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377623+0000", 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377622+0000", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377626+0000", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4350000000000001e-06 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377638+0000", 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2216e-05 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377653+0000", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5114e-05 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420629+0000", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042976331 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420699+0000", 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9349000000000003e-05 2024-09-21T15:34:20.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420762+0000", 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3812000000000005e-05 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124522 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 3997696~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.294666821, 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043278491000000002, 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124522 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377662+0000", 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377661+0000", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377665+0000", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377675+0000", 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0621999999999999e-05 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377695+0000", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9592000000000001e-05 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420796+0000", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043101511000000002 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420877+0000", 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0694999999999999e-05 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420940+0000", 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3170999999999995e-05 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:124523 3.1b 3:db6c0f42:::rbd_data.11914be32c41.0000000000000001:1e [read 4087808~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 12.294551122, 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043336070999999997, 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:20.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 124523 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:20.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377778+0000", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377702+0000", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999242 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377782+0000", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0505000000000003e-05 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:20.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377790+0000", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2740000000000007e-06 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.377796+0000", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6540000000000004e-06 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.420979+0000", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.043182812000000001 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421048+0000", 2024-09-21T15:34:20.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9449999999999994e-05 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.421114+0000", 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5278999999999996e-05 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:20.758 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:20.759 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:20.759 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:20.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:20.760 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:20.761 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:20.761 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:20.762 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:20.754+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:20.849 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:20.849 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:20.849 INFO:teuthology.orchestra.run.smithi107.stdout:4436 writesame 0x2c68756 thru 0x2c765ff data_size 0xb(0xdeaa bytes) 2024-09-21T15:34:20.861 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:20.860+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:20.867 INFO:teuthology.orchestra.run.smithi107.stdout:4438 trunc from 0xd6fa2e6 to 0xaedd9d4 2024-09-21T15:34:20.891 INFO:teuthology.orchestra.run.smithi107.stdout:4439 trunc from 0xaedd9d4 to 0xc8031c1 2024-09-21T15:34:20.897 INFO:teuthology.orchestra.run.smithi107.stdout:4440 writesame 0x80f4c07 thru 0x80feb39 data_size 0xa5(0x9f33 bytes) 2024-09-21T15:34:20.905 INFO:teuthology.orchestra.run.smithi107.stdout:4441 read 0x8b7388e thru 0x8b7871b (0x4e8e bytes) 2024-09-21T15:34:20.906 INFO:teuthology.orchestra.run.smithi107.stdout:4442 read 0x88f3540 thru 0x88f4c27 (0x16e8 bytes) 2024-09-21T15:34:20.906 INFO:teuthology.orchestra.run.smithi107.stdout:4443 writesame 0x5d8e66f thru 0x5d8f22f data_size 0x11(0xbc1 bytes) 2024-09-21T15:34:20.912 INFO:teuthology.orchestra.run.smithi107.stdout:4444 punch from 0x808e297 to 0x8093ff8, (0x5d61 bytes) 2024-09-21T15:34:20.912 INFO:teuthology.orchestra.run.smithi107.stdout:4445 punch from 0x53b25e6 to 0x53bc29c, (0x9cb6 bytes) 2024-09-21T15:34:20.912 INFO:teuthology.orchestra.run.smithi107.stdout:4446 read 0xabc521c thru 0xabc5451 (0x236 bytes) 2024-09-21T15:34:20.913 INFO:teuthology.orchestra.run.smithi107.stdout:4447 read 0x37b4f75 thru 0x37be3dd (0x9469 bytes) 2024-09-21T15:34:20.913 INFO:teuthology.orchestra.run.smithi107.stdout:4449 punch from 0x678c76c to 0x6798ae4, (0xc378 bytes) 2024-09-21T15:34:20.913 INFO:teuthology.orchestra.run.smithi107.stdout:4450 read 0x8f854e9 thru 0x8f8c3e9 (0x6f01 bytes) 2024-09-21T15:34:20.913 INFO:teuthology.orchestra.run.smithi107.stdout:4451 read 0x8ee675c thru 0x8eeb22b (0x4ad0 bytes) 2024-09-21T15:34:20.914 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:20.914 INFO:teuthology.orchestra.run.smithi107.stdout:4454 write 0x2671f56 thru 0x2674df9 (0x2ea4 bytes) 2024-09-21T15:34:20.920 INFO:teuthology.orchestra.run.smithi107.stdout:4455 write 0xb866fea thru 0xb876dbb (0xfdd2 bytes) 2024-09-21T15:34:20.934 INFO:teuthology.orchestra.run.smithi107.stdout:4456 read 0x445d12 thru 0x446751 (0xa40 bytes) 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:4457 read 0x52927c1 thru 0x529a3aa (0x7bea bytes) 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:4461 read 0x7a7f4a3 thru 0x7a82f62 (0x3ac0 bytes) 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:4462 punch from 0xdd3c8b to 0xdd6ad6, (0x2e4b bytes) 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:4464 punch from 0x63970e8 to 0x6399cb0, (0x2bc8 bytes) 2024-09-21T15:34:20.935 INFO:teuthology.orchestra.run.smithi107.stdout:4466 read 0x8aad33f thru 0x8ab6436 (0x90f8 bytes) 2024-09-21T15:34:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:4468 writesame 0xcdc7826 thru 0xcdd11d9 data_size 0x1(0x99b4 bytes) 2024-09-21T15:34:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:4469 write 0x57d6617 thru 0x57e3a2e (0xd418 bytes) 2024-09-21T15:34:20.957 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:20.957 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:20.957 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:20.957 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:20.961 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:20.958+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:20.973 INFO:teuthology.orchestra.run.smithi107.stdout:4470 trunc from 0xcdd11da to 0x1681a81 2024-09-21T15:34:20.981 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:21.023 INFO:teuthology.orchestra.run.smithi107.stdout:4471 trunc from 0x1681a81 to 0x2494adf 2024-09-21T15:34:21.032 INFO:teuthology.orchestra.run.smithi107.stdout:4473 read 0x1aa8b98 thru 0x1ab779e (0xec07 bytes) 2024-09-21T15:34:21.032 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.032 INFO:teuthology.orchestra.run.smithi107.stdout:4476 read 0x1f12fa7 thru 0x1f16def (0x3e49 bytes) 2024-09-21T15:34:21.032 INFO:teuthology.orchestra.run.smithi107.stdout:4477 read 0xad35f thru 0xb6a15 (0x96b7 bytes) 2024-09-21T15:34:21.034 INFO:teuthology.orchestra.run.smithi107.stdout:4478 punch from 0x1f73f2c to 0x1f7eef7, (0xafcb bytes) 2024-09-21T15:34:21.054 INFO:teuthology.orchestra.run.smithi107.stdout:4481 write 0xa494186 thru 0xa4999e9 (0x5864 bytes) 2024-09-21T15:34:21.062 INFO:teuthology.orchestra.run.smithi107.stdout:4482 read 0x721083c thru 0x7216589 (0x5d4e bytes) 2024-09-21T15:34:21.062 INFO:teuthology.orchestra.run.smithi107.stdout:4484 read 0x73579a3 thru 0x735ce93 (0x54f1 bytes) 2024-09-21T15:34:21.062 INFO:teuthology.orchestra.run.smithi107.stdout:4487 punch from 0x7c1a38d to 0x7c204ab, (0x611e bytes) 2024-09-21T15:34:21.062 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:21.058+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:21.073 INFO:teuthology.orchestra.run.smithi107.stdout:4488 writesame 0xdaf4154 thru 0xdafd73e data_size 0xb(0x95eb bytes) 2024-09-21T15:34:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:4489 punch from 0x72b9886 to 0x72c70f7, (0xd871 bytes) 2024-09-21T15:34:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:4491 write 0x8936456 thru 0x8944878 (0xe423 bytes) 2024-09-21T15:34:21.080 INFO:teuthology.orchestra.run.smithi107.stdout:4492 read 0x278db41 thru 0x2799fa8 (0xc468 bytes) 2024-09-21T15:34:21.080 INFO:teuthology.orchestra.run.smithi107.stdout:4493 punch from 0x7e42e26 to 0x7e482fa, (0x54d4 bytes) 2024-09-21T15:34:21.080 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.080 INFO:teuthology.orchestra.run.smithi107.stdout:4495 writesame 0x4c78d3b thru 0x4c83a78 data_size 0x19(0xad3e bytes) 2024-09-21T15:34:21.088 INFO:teuthology.orchestra.run.smithi107.stdout:4496 read 0x9fb319d thru 0x9fb8af3 (0x5957 bytes) 2024-09-21T15:34:21.088 INFO:teuthology.orchestra.run.smithi107.stdout:4497 writesame 0xa8b0937 thru 0xa8b9163 data_size 0x1(0x882d bytes) 2024-09-21T15:34:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:4498 punch from 0x9d13174 to 0x9d20cec, (0xdb78 bytes) 2024-09-21T15:34:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:4499 read 0x1478cb3 thru 0x147ca32 (0x3d80 bytes) 2024-09-21T15:34:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:4502 read 0x20515df thru 0x2056af6 (0x5518 bytes) 2024-09-21T15:34:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:4503 write 0xb73054d thru 0xb73f434 (0xeee8 bytes) 2024-09-21T15:34:21.121 INFO:teuthology.orchestra.run.smithi107.stdout:4504 write 0x7c59224 thru 0x7c5b617 (0x23f4 bytes) 2024-09-21T15:34:21.127 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.127 INFO:teuthology.orchestra.run.smithi107.stdout:4506 punch from 0x201a165 to 0x2025a05, (0xb8a0 bytes) 2024-09-21T15:34:21.127 INFO:teuthology.orchestra.run.smithi107.stdout:4507 read 0x356a881 thru 0x3574174 (0x98f4 bytes) 2024-09-21T15:34:21.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:34:21.127 INFO:teuthology.orchestra.run.smithi107.stdout:4508 write 0xac5fc08 thru 0xac62f63 (0x335c bytes) 2024-09-21T15:34:21.139 INFO:teuthology.orchestra.run.smithi107.stdout:4510 write 0x72643df thru 0x7272e5a (0xea7c bytes) 2024-09-21T15:34:21.148 INFO:teuthology.orchestra.run.smithi107.stdout:4511 trunc from 0xdafd73f to 0xd06369b 2024-09-21T15:34:21.154 INFO:teuthology.orchestra.run.smithi107.stdout:4512 read 0x52d545a thru 0x52e50f7 (0xfc9e bytes) 2024-09-21T15:34:21.154 INFO:teuthology.orchestra.run.smithi107.stdout:4513 read 0xa12193 thru 0xa1c5f0 (0xa45e bytes) 2024-09-21T15:34:21.156 INFO:teuthology.orchestra.run.smithi107.stdout:4514 punch from 0x657ca35 to 0x657f44f, (0x2a1a bytes) 2024-09-21T15:34:21.156 INFO:teuthology.orchestra.run.smithi107.stdout:4515 writesame 0x16f8490 thru 0x16fedf6 data_size 0xb(0x6967 bytes) 2024-09-21T15:34:21.159 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:21.159 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:21.161 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.161 INFO:teuthology.orchestra.run.smithi107.stdout:4519 read 0xbb64536 thru 0xbb7060e (0xc0d9 bytes) 2024-09-21T15:34:21.161 INFO:teuthology.orchestra.run.smithi107.stdout:4520 write 0xab7a738 thru 0xab85fa9 (0xb872 bytes) 2024-09-21T15:34:21.163 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:21.158+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:21.168 INFO:teuthology.orchestra.run.smithi107.stdout:4521 writesame 0x451e437 thru 0x452c443 data_size 0x3(0xe00d bytes) 2024-09-21T15:34:21.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:21.181 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.181 INFO:teuthology.orchestra.run.smithi107.stdout:4523 writesame 0x52d5282 thru 0x52dc252 data_size 0x19(0x6fd1 bytes) 2024-09-21T15:34:21.187 INFO:teuthology.orchestra.run.smithi107.stdout:4524 read 0x3fa785e thru 0x3facb46 (0x52e9 bytes) 2024-09-21T15:34:21.195 INFO:teuthology.orchestra.run.smithi107.stdout:4526 trunc from 0xd06369b to 0x80acbd3 2024-09-21T15:34:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:4528 read 0x2bc3716 thru 0x2bc3de6 (0x6d1 bytes) 2024-09-21T15:34:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:4529 read 0x4974429 thru 0x497c3c2 (0x7f9a bytes) 2024-09-21T15:34:21.224 INFO:teuthology.orchestra.run.smithi107.stdout:4530 writesame 0xa78213d thru 0xa7880a0 data_size 0x16(0x5f64 bytes) 2024-09-21T15:34:21.225 INFO:teuthology.orchestra.run.smithi107.stdout:4531 read 0x83a390 thru 0x849fea (0xfc5b bytes) 2024-09-21T15:34:21.229 INFO:teuthology.orchestra.run.smithi107.stdout:4533 trunc from 0xa7880a1 to 0x91ce312 2024-09-21T15:34:21.235 INFO:teuthology.orchestra.run.smithi107.stdout:4534 read 0x29c0933 thru 0x29c915f (0x882d bytes) 2024-09-21T15:34:21.235 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.246 INFO:teuthology.orchestra.run.smithi107.stdout:4536 write 0xbdc52fe thru 0xbdd2278 (0xcf7b bytes) 2024-09-21T15:34:21.255 INFO:teuthology.orchestra.run.smithi107.stdout:4539 read 0x4c17c3b thru 0x4c1959e (0x1964 bytes) 2024-09-21T15:34:21.255 INFO:teuthology.orchestra.run.smithi107.stdout:4541 write 0x31f322a thru 0x3201b88 (0xe95f bytes) 2024-09-21T15:34:21.264 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:21.262+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:21.264 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.269 INFO:teuthology.orchestra.run.smithi107.stdout:4544 trunc from 0xbdd2279 to 0x7e73786 2024-09-21T15:34:21.283 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:21.297 INFO:teuthology.orchestra.run.smithi107.stdout:4546 write 0xde11b6b thru 0xde1b32c (0x97c2 bytes) 2024-09-21T15:34:21.303 INFO:teuthology.orchestra.run.smithi107.stdout:4547 write 0x3d492e5 thru 0x3d4b746 (0x2462 bytes) 2024-09-21T15:34:21.308 INFO:teuthology.orchestra.run.smithi107.stdout:4548 read 0xa114ad2 thru 0xa11a871 (0x5da0 bytes) 2024-09-21T15:34:21.308 INFO:teuthology.orchestra.run.smithi107.stdout:4550 read 0x9e345b thru 0x9e9974 (0x651a bytes) 2024-09-21T15:34:21.309 INFO:teuthology.orchestra.run.smithi107.stdout:4551 write 0x7487c79 thru 0x749546f (0xd7f7 bytes) 2024-09-21T15:34:21.321 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.198111944000004, 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:21.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:21.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.326 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.328 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.755955731, 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.330 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.332 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.755930984999999, 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.333 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:21.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.336 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.755886035000003, 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.337 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.339 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.752029376999999, 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.343 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751985017000003, 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751931667999997, 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:21.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:21.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751900274999997, 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:21.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:21.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:21.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.356 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751854549000001, 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:21.358 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:21.359 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.360 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751825175999997, 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:21.361 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:21.362 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:21.363 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.364 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751800615999997, 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:21.365 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.366 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.751773978999999, 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.369 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.370 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.371 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.372 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747935411999997, 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.373 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.375 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.376 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747905592000002, 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.377 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.378 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:21.379 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:21.380 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747880062, 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:21.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.382 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.383 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:21.384 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747854355999998, 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:21.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.386 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:21.387 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:21.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747804316, 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.389 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:21.390 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.391 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.392 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747769515999998, 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:21.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.394 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.395 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.397 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.747744867000002, 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:21.399 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:21.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.401 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 41.74369446, 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:21.402 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:21.403 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:21.404 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:21.407 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:21.407 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:21.404+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:21.508 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:21.506+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3.3e917af4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.25280391499999999, 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.25284631499999999, 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "waiting for peered", 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 1 2024-09-21T15:34:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:21.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:21.608 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:21.604+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:21.709 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:21.706+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:21.748 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:21.748 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:21.762 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:21.810 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:21.808+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:21.910 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:21.906+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3324 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'19, mlcod=30'17)", 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.895994670999997, 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037253210000000002, 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024648+0000", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024646+0000", 2024-09-21T15:34:21.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024659+0000", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2639999999999999e-05 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024675+0000", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6144999999999999e-05 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024686+0000", 2024-09-21T15:34:21.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0723000000000001e-05 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024741+0000", 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5569000000000003e-05 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.024760+0000", 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8564e-05 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028329+0000", 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035689369999999999 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.028373+0000", 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4472999999999999e-05 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3358 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'20, mlcod=30'18)", 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:21.927 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.882040630999995, 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037648740000000001, 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038602+0000", 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038600+0000", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038612+0000", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1428e-05 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038628+0000", 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6022999999999999e-05 2024-09-21T15:34:21.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038638+0000", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7750000000000004e-06 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038692+0000", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4203000000000003e-05 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.038718+0000", 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6591999999999999e-05 2024-09-21T15:34:21.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042329+0000", 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036102579999999999 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.042367+0000", 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8182000000000001e-05 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.873127087, 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3516 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'31, mlcod=30'29)", 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.782199835, 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0036669929999999999, 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138443+0000", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138442+0000", 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138451+0000", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138464+0000", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3543999999999999e-05 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138473+0000", 2024-09-21T15:34:21.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.302e-06 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138564+0000", 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0783000000000001e-05 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.138597+0000", 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2428999999999997e-05 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142081+0000", 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034848489999999999 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.142110+0000", 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8266e-05 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:21.940 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.822813881000002, 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:21.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:21.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7323 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'204, mlcod=36'199)", 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.821847954000006, 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041498849999999999, 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098795+0000", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098794+0000", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098800+0000", 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5450000000000003e-06 2024-09-21T15:34:21.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098808+0000", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.85e-06 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098813+0000", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7099999999999998e-06 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098843+0000", 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9759000000000001e-05 2024-09-21T15:34:21.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.098856+0000", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2994e-05 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102933+0000", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004077298 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102945+0000", 2024-09-21T15:34:21.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1294000000000001e-05 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.811643971999999, 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62269 3.0 e68/50 3:072ab387:::rbd_header.1191e6bca1cd:head v 68'364, mlcod=68'362)", 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 33.042102276000001, 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037514879999999999, 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878540+0000", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878538+0000", 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:21.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878557+0000", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9173000000000001e-05 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878592+0000", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5429000000000002e-05 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878610+0000", 2024-09-21T15:34:21.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7974000000000001e-05 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878698+0000", 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7293000000000006e-05 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.878730+0000", 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2175000000000003e-05 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882213+0000", 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034834689999999999 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.882292+0000", 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8453000000000003e-05 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:62282 3.0 e69/50 3:01c4b5a3:::rbd_data.1191e6bca1cd.00000000000000da:head v 69'365, mlcod=68'363)", 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 33.017976988000001, 2024-09-21T15:34:21.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037099500000000001, 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902666+0000", 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902665+0000", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902697+0000", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1680000000000002e-05 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902716+0000", 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9145999999999999e-05 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902726+0000", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0139000000000001e-05 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902782+0000", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5986999999999998e-05 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.902801+0000", 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8300000000000001e-05 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906328+0000", 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0035275829999999999 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:48.906376+0000", 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7367000000000002e-05 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.961 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:102949 3.4 e72/47 3:221b15ed:::rbd_data.1191b30e35a1.0000000000000042:head v 72'106, mlcod=72'104)", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.584063654000001, 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040397410000000003, 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336579+0000", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336577+0000", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336598+0000", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1659e-05 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336622+0000", 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3204e-05 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336633+0000", 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0967e-05 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336686+0000", 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3915999999999997e-05 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.336712+0000", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5060999999999999e-05 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340579+0000", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038670309999999999 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:57.340619+0000", 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0080999999999998e-05 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.965 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 13.396082675000001, 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:153989 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'491, mlcod=88'487)", 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:20.924741+0000", 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.99590193599999999, 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043953209999999998, 2024-09-21T15:34:21.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924741+0000", 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924741+0000", 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924740+0000", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924747+0000", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9079999999999998e-06 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924759+0000", 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2649e-05 2024-09-21T15:34:21.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924767+0000", 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1590000000000004e-06 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924809+0000", 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2519999999999999e-05 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.924830+0000", 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1121999999999999e-05 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.929106+0000", 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042762679999999997 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:20.929136+0000", 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9470000000000001e-05 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154138 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'504, mlcod=88'500)", 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.001493+0000", 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91914976900000001, 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041851570000000001, 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001493+0000", 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001493+0000", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001491+0000", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001503+0000", 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1249e-05 2024-09-21T15:34:21.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001523+0000", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9947999999999999e-05 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001533+0000", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0156999999999999e-05 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001572+0000", 2024-09-21T15:34:21.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8828999999999997e-05 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.001583+0000", 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1868000000000001e-05 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005635+0000", 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0040514690000000003 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005678+0000", 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3120000000000001e-05 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154140 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'505, mlcod=88'500)", 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91787671000000004, 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061868679999999999, 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002765+0000", 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002772+0000", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1799999999999999e-06 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002785+0000", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2716e-05 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002795+0000", 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7510000000000003e-06 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002835+0000", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0129000000000002e-05 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002855+0000", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9741e-05 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008875+0000", 2024-09-21T15:34:21.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060205340000000001 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008953+0000", 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7595999999999994e-05 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154141 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'506, mlcod=88'501)", 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91610343299999997, 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044846160000000003, 2024-09-21T15:34:21.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004538+0000", 2024-09-21T15:34:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004545+0000", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999998e-06 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004558+0000", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2847000000000001e-05 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004565+0000", 2024-09-21T15:34:21.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0280000000000003e-06 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004600+0000", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4774999999999998e-05 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004612+0000", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2398000000000001e-05 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008961+0000", 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043488329999999999 2024-09-21T15:34:21.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.009024+0000", 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2484999999999996e-05 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154143 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'507, mlcod=88'501)", 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91490458500000005, 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053764679999999997, 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005737+0000", 2024-09-21T15:34:21.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005744+0000", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0210000000000002e-06 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005758+0000", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005765+0000", 2024-09-21T15:34:21.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5180000000000001e-06 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005854+0000", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9349000000000001e-05 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005873+0000", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814e-05 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011030+0000", 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051572900000000001 2024-09-21T15:34:21.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011114+0000", 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4271999999999995e-05 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154144 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'508, mlcod=88'503)", 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.007354+0000", 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91328860199999995, 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038240470000000001, 2024-09-21T15:34:21.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007354+0000", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007354+0000", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007353+0000", 2024-09-21T15:34:21.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007360+0000", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.494e-06 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007372+0000", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2187e-05 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007380+0000", 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0749999999999998e-06 2024-09-21T15:34:21.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007471+0000", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1594000000000004e-05 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.007523+0000", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1579999999999997e-05 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011120+0000", 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003596667 2024-09-21T15:34:21.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011178+0000", 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8285000000000002e-05 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154146 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'509, mlcod=88'503)", 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.91247233699999997, 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055853480000000004, 2024-09-21T15:34:21.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008169+0000", 2024-09-21T15:34:21.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008177+0000", 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8790000000000003e-06 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008190+0000", 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2362e-05 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:21.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008198+0000", 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1340000000000004e-06 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008440+0000", 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024192699999999999 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008459+0000", 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9655000000000001e-05 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013713+0000", 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052535560000000004 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013756+0000", 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2608999999999998e-05 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154315 3.0 e89/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 89'562, mlcod=89'558)", 2024-09-21T15:34:21.999 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.258544+0000", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.66209850800000003, 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038592050000000001, 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258544+0000", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258544+0000", 2024-09-21T15:34:22.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258543+0000", 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258551+0000", 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3289999999999994e-06 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258567+0000", 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5437000000000001e-05 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258576+0000", 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1910000000000006e-06 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258614+0000", 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8311e-05 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.258627+0000", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2325e-05 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.262378+0000", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037517010000000001 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.262403+0000", 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4935000000000001e-05 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.22818881199999999, 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056653390000000001, 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4204", 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.170:0/3270855337", 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.004 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692450+0000", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692472+0000", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1977e-05 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692511+0000", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8708999999999998e-05 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692528+0000", 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7640000000000001e-05 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.697842+0000", 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053131849999999998 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.698119+0000", 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027754100000000002 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.008 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.008 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:22.009 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:22.011 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:22.008+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:22.112 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:22.110+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:22.145 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:34:22.151 INFO:teuthology.orchestra.run.smithi107.stdout:4552 write 0xc7cf570 thru 0xc7decb2 (0xf743 bytes) 2024-09-21T15:34:22.161 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:22.172 INFO:teuthology.orchestra.run.smithi107.stdout:4554 write 0xe27231e thru 0xe279179 (0x6e5c bytes) 2024-09-21T15:34:22.177 INFO:teuthology.orchestra.run.smithi107.stdout:4555 clone 34 order 19 su 65536 sc 12 2024-09-21T15:34:22.188 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:22.188 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:22.188 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:22.188 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:22.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:22.244 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:22.242+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:22.314 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:22.312+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:22.359 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:22.359 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:22.359 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:22.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:22.420 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:22.418+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 90.127742623000003, 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:22.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 90.099531447000004, 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:22.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:22.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:22.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.910232078999996, 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:22.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:22.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:22.513 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.778847388000003, 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:22.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.517 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.617749984, 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:22.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:22.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:22.523 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.617697437000004, 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:22.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:22.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:22.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.617649311999998, 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:22.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.605252424, 2024-09-21T15:34:22.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:22.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:22.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.044479441999997, 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.540 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:22.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.545 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.977455075999998, 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:22.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:22.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.842334945000005, 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:22.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:22.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:22.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:22.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:22.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.556 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.801672801999999, 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.560 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.801659584999996, 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.561 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:22.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:22.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:22.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.801646778000006, 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:22.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.801606371000005, 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.801581902999999, 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:22.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.577 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.782359915000001, 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.695056332000007, 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:22.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:22.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:22.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.700370889, 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:22.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 15.286111542, 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:22.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:22.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:22.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:22.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:22.597 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:22.598 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:22.599 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:22.594+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:22.699 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:22.698+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:22.730 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:22.730 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:22.730 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:22.730 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:22.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:22.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:22.800 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:22.796+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:22.871 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:22.871 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:22.871 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:22.871 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:22.871 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:22.885 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:22.927 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:22.927 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:22.929 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:22.928+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:22.944 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:22.944 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:22.960 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:22.960 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:22.976 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:22.977 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:22.994 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:22.994 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:23.001 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:23.000+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:23.012 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:23.012 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:23.030 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:23.030 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:23.051 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:23.053 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.053 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:23.109 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:23.108+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:23.120 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:23.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T15:34:23.203 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:23.200+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:23.233 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.233 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:23.263 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone33 2024-09-21T15:34:23.341 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:23.340+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:23.404 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:23.400+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:23.413 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.413 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:23.506 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:23.502+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:23.522 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T15:34:23.534 INFO:teuthology.orchestra.run.smithi107.stdout:4556 read 0xc3948ad thru 0xc394f27 (0x67b bytes) 2024-09-21T15:34:23.535 INFO:teuthology.orchestra.run.smithi107.stdout:4557 punch from 0xb77ebb8 to 0xb782197, (0x35df bytes) 2024-09-21T15:34:23.542 INFO:teuthology.orchestra.run.smithi107.stdout:4559 read 0x9b22000 thru 0x9b2cde8 (0xade9 bytes) 2024-09-21T15:34:23.542 INFO:teuthology.orchestra.run.smithi107.stdout:4560 write 0x464c4bb thru 0x464ce41 (0x987 bytes) 2024-09-21T15:34:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:4561 read 0xb5c2e79 thru 0xb5c55e8 (0x2770 bytes) 2024-09-21T15:34:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:4565 punch from 0x2f6ba41 to 0x2f741ca, (0x8789 bytes) 2024-09-21T15:34:23.549 INFO:teuthology.orchestra.run.smithi107.stdout:4566 writesame 0x39b8cd1 thru 0x39bea95 data_size 0x1(0x5dc5 bytes) 2024-09-21T15:34:23.585 INFO:teuthology.orchestra.run.smithi107.stdout:4567 read 0x31b18ca thru 0x31b7d4d (0x6484 bytes) 2024-09-21T15:34:23.585 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.585 INFO:teuthology.orchestra.run.smithi107.stdout:4569 read 0x1a23994 thru 0x1a2f8f4 (0xbf61 bytes) 2024-09-21T15:34:23.585 INFO:teuthology.orchestra.run.smithi107.stdout:4570 write 0xdc841a9 thru 0xdc8bbc5 (0x7a1d bytes) 2024-09-21T15:34:23.592 INFO:teuthology.orchestra.run.smithi107.stdout:4571 read 0x5de97e8 thru 0x5df95f3 (0xfe0c bytes) 2024-09-21T15:34:23.592 INFO:teuthology.orchestra.run.smithi107.stdout:4572 write 0x629df85 thru 0x62a9f1f (0xbf9b bytes) 2024-09-21T15:34:23.593 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.593 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:4573 read 0x5c149fa thru 0x5c1c9c4 (0x7fcb bytes) 2024-09-21T15:34:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:4574 read 0xd6dc706 thru 0xd6ea38e (0xdc89 bytes) 2024-09-21T15:34:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:4575 read 0x8d5a6d5 thru 0x8d66fc5 (0xc8f1 bytes) 2024-09-21T15:34:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:4576 punch from 0x8e57a51 to 0x8e5922b, (0x17da bytes) 2024-09-21T15:34:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:4577 write 0x917cc16 thru 0x9186551 (0x993c bytes) 2024-09-21T15:34:23.607 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:23.604+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:23.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:23.610 INFO:teuthology.orchestra.run.smithi107.stdout:4579 punch from 0x6642ace to 0x664df71, (0xb4a3 bytes) 2024-09-21T15:34:23.610 INFO:teuthology.orchestra.run.smithi107.stdout:4580 writesame 0x52c7c76 thru 0x52ca18e data_size 0x1(0x2519 bytes) 2024-09-21T15:34:23.627 INFO:teuthology.orchestra.run.smithi107.stdout:4581 read 0xd1fa762 thru 0xd2075ca (0xce69 bytes) 2024-09-21T15:34:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:4583 write 0xe02d92 thru 0xe1104e (0xe2bd bytes) 2024-09-21T15:34:23.634 INFO:teuthology.orchestra.run.smithi107.stdout:4584 read 0x387cba6 thru 0x3880b64 (0x3fbf bytes) 2024-09-21T15:34:23.634 INFO:teuthology.orchestra.run.smithi107.stdout:4585 write 0x125ccdb thru 0x12648a9 (0x7bcf bytes) 2024-09-21T15:34:23.645 INFO:teuthology.orchestra.run.smithi107.stdout:4586 read 0x95b9ae1 thru 0x95c2eda (0x93fa bytes) 2024-09-21T15:34:23.645 INFO:teuthology.orchestra.run.smithi107.stdout:4587 write 0x256eb8 thru 0x25d065 (0x61ae bytes) 2024-09-21T15:34:23.654 INFO:teuthology.orchestra.run.smithi107.stdout:4589 write 0x1c89e4a thru 0x1c96035 (0xc1ec bytes) 2024-09-21T15:34:23.663 INFO:teuthology.orchestra.run.smithi107.stdout:4590 write 0x7001b53 thru 0x7007522 (0x59d0 bytes) 2024-09-21T15:34:23.668 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.668 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.668 INFO:teuthology.orchestra.run.smithi107.stdout:4595 punch from 0x63d4faf to 0x63dbdde, (0x6e2f bytes) 2024-09-21T15:34:23.668 INFO:teuthology.orchestra.run.smithi107.stdout:4596 read 0x60c7643 thru 0x60cfb6e (0x852c bytes) 2024-09-21T15:34:23.668 INFO:teuthology.orchestra.run.smithi107.stdout:4599 writesame 0xc6ca5c8 thru 0xc6cd8a4 data_size 0x1(0x32dd bytes) 2024-09-21T15:34:23.693 INFO:teuthology.orchestra.run.smithi107.stdout:4600 trunc from 0xe27917a to 0xd389fbd 2024-09-21T15:34:23.708 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:23.706+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:23.734 INFO:teuthology.orchestra.run.smithi107.stdout:4601 read 0x4b87e27 thru 0x4b8dc6a (0x5e44 bytes) 2024-09-21T15:34:23.734 INFO:teuthology.orchestra.run.smithi107.stdout:4605 punch from 0x7c2ed99 to 0x7c3e83f, (0xfaa6 bytes) 2024-09-21T15:34:23.734 INFO:teuthology.orchestra.run.smithi107.stdout:4606 read 0x3203c14 thru 0x320e9b2 (0xad9f bytes) 2024-09-21T15:34:23.735 INFO:teuthology.orchestra.run.smithi107.stdout:4607 read 0xc4f3375 thru 0xc4f78bc (0x4548 bytes) 2024-09-21T15:34:23.735 INFO:teuthology.orchestra.run.smithi107.stdout:4609 punch from 0x3e65f72 to 0x3e711ee, (0xb27c bytes) 2024-09-21T15:34:23.735 INFO:teuthology.orchestra.run.smithi107.stdout:4610 read 0x862ae24 thru 0x86323b3 (0x7590 bytes) 2024-09-21T15:34:23.735 INFO:teuthology.orchestra.run.smithi107.stdout:4612 write 0xc826eb3 thru 0xc829de9 (0x2f37 bytes) 2024-09-21T15:34:23.743 INFO:teuthology.orchestra.run.smithi107.stdout:4616 writesame 0x1fc8f53 thru 0x1fd6583 data_size 0x1(0xd631 bytes) 2024-09-21T15:34:23.786 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.786 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.786 INFO:teuthology.orchestra.run.smithi107.stdout:4617 write 0xb204191 thru 0xb20f9c3 (0xb833 bytes) 2024-09-21T15:34:23.793 INFO:teuthology.orchestra.run.smithi107.stdout:4618 punch from 0x91f2483 to 0x91f2486, (0x3 bytes) 2024-09-21T15:34:23.793 INFO:teuthology.orchestra.run.smithi107.stdout:4620 read 0x4a397f5 thru 0x4a455ff (0xbe0b bytes) 2024-09-21T15:34:23.793 INFO:teuthology.orchestra.run.smithi107.stdout:4621 punch from 0x7e57c5f to 0x7e6113c, (0x94dd bytes) 2024-09-21T15:34:23.793 INFO:teuthology.orchestra.run.smithi107.stdout:4622 punch from 0x3fc1827 to 0x3fcd645, (0xbe1e bytes) 2024-09-21T15:34:23.794 INFO:teuthology.orchestra.run.smithi107.stdout:4624 write 0xc8ef7bb thru 0xc8fae4b (0xb691 bytes) 2024-09-21T15:34:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:4625 read 0x2c2b0a thru 0x2ced57 (0xc24e bytes) 2024-09-21T15:34:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:4628 read 0x39e9212 thru 0x39e95f6 (0x3e5 bytes) 2024-09-21T15:34:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:4629 read 0xbf7e300 thru 0xbf8961f (0xb320 bytes) 2024-09-21T15:34:23.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:23.833 INFO:teuthology.orchestra.run.smithi107.stdout:4630 trunc from 0xd389fbd to 0x2ce31b 2024-09-21T15:34:23.845 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:23.844+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:23.909 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:23.908+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:23.925 INFO:teuthology.orchestra.run.smithi107.stdout:4633 write 0xb851bc4 thru 0xb85736b (0x57a8 bytes) 2024-09-21T15:34:23.931 INFO:teuthology.orchestra.run.smithi107.stdout:4634 writesame 0x7136247 thru 0x713777a data_size 0x2e(0x1534 bytes) 2024-09-21T15:34:23.935 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:23.936 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:23.937 INFO:teuthology.orchestra.run.smithi107.stdout:4635 write 0xa22512d thru 0xa231050 (0xbf24 bytes) 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:4636 read 0x4dc9e5e thru 0x4dd2519 (0x86bc bytes) 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:4639 read 0x1cc480c thru 0x1cc49bd (0x1b2 bytes) 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:4640 read 0x313d432 thru 0x3149bb8 (0xc787 bytes) 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:4642 read 0x59b99a8 thru 0x59c3858 (0x9eb1 bytes) 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.946 INFO:teuthology.orchestra.run.smithi107.stdout:4644 write 0x14b1702 thru 0x14b90de (0x79dd bytes) 2024-09-21T15:34:23.952 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:23.953 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:23.953 INFO:teuthology.orchestra.run.smithi107.stdout:4646 punch from 0x40728df to 0x407d68f, (0xadb0 bytes) 2024-09-21T15:34:23.953 INFO:teuthology.orchestra.run.smithi107.stdout:4647 read 0x7538557 thru 0x7542b1a (0xa5c4 bytes) 2024-09-21T15:34:23.968 INFO:teuthology.orchestra.run.smithi107.stdout:4648 trunc from 0xb85736c to 0x5ea65a1 2024-09-21T15:34:23.996 INFO:teuthology.orchestra.run.smithi107.stdout:4649 trunc from 0x5ea65a1 to 0x87d567c 2024-09-21T15:34:24.009 INFO:teuthology.orchestra.run.smithi107.stdout:4650 write 0x520109d thru 0x5205255 (0x41b9 bytes) 2024-09-21T15:34:24.010 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:24.006+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:24.016 INFO:teuthology.orchestra.run.smithi107.stdout:4651 write 0x6676032 thru 0x668227d (0xc24c bytes) 2024-09-21T15:34:24.040 INFO:teuthology.orchestra.run.smithi107.stdout:4653 trunc from 0x87d567c to 0x21eadeb 2024-09-21T15:34:24.062 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.093 INFO:teuthology.orchestra.run.smithi107.stdout:4658 trunc from 0x21eadeb to 0xe6b5d63 2024-09-21T15:34:24.108 INFO:teuthology.orchestra.run.smithi107.stdout:4659 read 0xc0b24e3 thru 0xc0b47a8 (0x22c6 bytes) 2024-09-21T15:34:24.108 INFO:teuthology.orchestra.run.smithi107.stdout:4660 read 0x4642133 thru 0x464d42b (0xb2f9 bytes) 2024-09-21T15:34:24.108 INFO:teuthology.orchestra.run.smithi107.stdout:4662 write 0xcd301cd thru 0xcd3f06d (0xeea1 bytes) 2024-09-21T15:34:24.110 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:24.106+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:24.116 INFO:teuthology.orchestra.run.smithi107.stdout:4663 writesame 0x8319ec2 thru 0x831ccbf data_size 0x1d(0x2dfe bytes) 2024-09-21T15:34:24.122 INFO:teuthology.orchestra.run.smithi107.stdout:4664 write 0x1246df6 thru 0x1253594 (0xc79f bytes) 2024-09-21T15:34:24.131 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.131 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.132 INFO:teuthology.orchestra.run.smithi107.stdout:4665 write 0xcef0fe2 thru 0xcef6740 (0x575f bytes) 2024-09-21T15:34:24.138 INFO:teuthology.orchestra.run.smithi107.stdout:4666 punch from 0x8bb7f58 to 0x8bc57ef, (0xd897 bytes) 2024-09-21T15:34:24.147 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:24.155 INFO:teuthology.orchestra.run.smithi107.stdout:4667 write 0xe9d83ba thru 0xe9e03e8 (0x802f bytes) 2024-09-21T15:34:24.157 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T15:34:24.166 INFO:teuthology.orchestra.run.smithi107.stdout:4668 read 0x1f5ff7a thru 0x1f63438 (0x34bf bytes) 2024-09-21T15:34:24.166 INFO:teuthology.orchestra.run.smithi107.stdout:4669 write 0x2f55d3e thru 0x2f64fda (0xf29d bytes) 2024-09-21T15:34:24.173 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.173 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:34:24.176 INFO:teuthology.orchestra.run.smithi107.stdout:4671 trunc from 0xe9e03e9 to 0xd0eb35d 2024-09-21T15:34:24.186 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T15:34:24.190 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.191 INFO:teuthology.orchestra.run.smithi107.stdout:4675 read 0x969e76b thru 0x96acd48 (0xe5de bytes) 2024-09-21T15:34:24.191 INFO:teuthology.orchestra.run.smithi107.stdout:4676 punch from 0x9c716db to 0x9c7d961, (0xc286 bytes) 2024-09-21T15:34:24.191 INFO:teuthology.orchestra.run.smithi107.stdout:4677 write 0x2d69960 thru 0x2d710c5 (0x7766 bytes) 2024-09-21T15:34:24.206 INFO:teuthology.orchestra.run.smithi107.stdout:4679 writesame 0xde1a29e thru 0xde29e1b data_size 0x1(0xfb7e bytes) 2024-09-21T15:34:24.206 INFO:teuthology.orchestra.run.smithi107.stdout:4681 writesame 0x1885813 thru 0x188d5d7 data_size 0x1(0x7dc5 bytes) 2024-09-21T15:34:24.211 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:24.206+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:24.229 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.229 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.229 INFO:teuthology.orchestra.run.smithi107.stdout:4684 writesame 0x8260197 thru 0x82690c1 data_size 0x13(0x8f2b bytes) 2024-09-21T15:34:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:4685 punch from 0x2211151 to 0x2215ed2, (0x4d81 bytes) 2024-09-21T15:34:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:4686 read 0x15f1b5c thru 0x15f2bda (0x107f bytes) 2024-09-21T15:34:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:4687 read 0xd020ada thru 0xd023137 (0x265e bytes) 2024-09-21T15:34:24.235 INFO:teuthology.orchestra.run.smithi107.stdout:4688 writesame 0xa8b893e thru 0xa8bc879 data_size 0x4c(0x3f3c bytes) 2024-09-21T15:34:24.239 INFO:teuthology.orchestra.run.smithi107.stdout:4689 write 0xaddbe2a thru 0xaddd008 (0x11df bytes) 2024-09-21T15:34:24.243 INFO:teuthology.orchestra.run.smithi107.stdout:4691 read 0x298f9b7 thru 0x29995c2 (0x9c0c bytes) 2024-09-21T15:34:24.244 INFO:teuthology.orchestra.run.smithi107.stdout:4692 writesame 0x9949fac thru 0x994b139 data_size 0x7(0x118e bytes) 2024-09-21T15:34:24.249 INFO:teuthology.orchestra.run.smithi107.stdout:4694 write 0x2850de7 thru 0x285a0ca (0x92e4 bytes) 2024-09-21T15:34:24.254 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.254 INFO:teuthology.orchestra.run.smithi107.stdout:4696 write 0xce0cd66 thru 0xce14ec3 (0x815e bytes) 2024-09-21T15:34:24.263 INFO:teuthology.orchestra.run.smithi107.stdout:4697 read 0x37ba16d thru 0x37bc0e8 (0x1f7c bytes) 2024-09-21T15:34:24.263 INFO:teuthology.orchestra.run.smithi107.stdout:4699 punch from 0x9d6c677 to 0x9d76dbb, (0xa744 bytes) 2024-09-21T15:34:24.263 INFO:teuthology.orchestra.run.smithi107.stdout:4700 punch from 0x3e04ecb to 0x3e13f8b, (0xf0c0 bytes) 2024-09-21T15:34:24.263 INFO:teuthology.orchestra.run.smithi107.stdout:4701 read 0x675b6f3 thru 0x676a2b5 (0xebc3 bytes) 2024-09-21T15:34:24.263 INFO:teuthology.orchestra.run.smithi107.stdout:4702 writesame 0x540bd00 thru 0x541039d data_size 0x45(0x469e bytes) 2024-09-21T15:34:24.278 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.278 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.285 INFO:teuthology.orchestra.run.smithi107.stdout:4704 trunc from 0xde29e1c to 0x4738c58 2024-09-21T15:34:24.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:4705 punch from 0x4391eca to 0x4393e00, (0x1f36 bytes) 2024-09-21T15:34:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:4707 read 0x31929af thru 0x31a2063 (0xf6b5 bytes) 2024-09-21T15:34:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:4708 write 0x454f332 thru 0x4551c10 (0x28df bytes) 2024-09-21T15:34:24.310 INFO:teuthology.orchestra.run.smithi107.stdout:4709 punch from 0x41136ca to 0x411b9a2, (0x82d8 bytes) 2024-09-21T15:34:24.310 INFO:teuthology.orchestra.run.smithi107.stdout:4710 read 0x3399994 thru 0x339a6f4 (0xd61 bytes) 2024-09-21T15:34:24.311 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:24.306+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:4711 write 0x63985d4 thru 0x639937c (0xda9 bytes) 2024-09-21T15:34:24.323 INFO:teuthology.orchestra.run.smithi107.stdout:4712 writesame 0x5b778ce thru 0x5b7b665 data_size 0x24(0x3d98 bytes) 2024-09-21T15:34:24.327 INFO:teuthology.orchestra.run.smithi107.stdout:4713 writesame 0x2d32fd4 thru 0x2d3c33d data_size 0x1(0x936a bytes) 2024-09-21T15:34:24.355 INFO:teuthology.orchestra.run.smithi107.stdout:4714 read 0x5b15ddb thru 0x5b17997 (0x1bbd bytes) 2024-09-21T15:34:24.373 INFO:teuthology.orchestra.run.smithi107.stdout:4715 write 0xda43ef6 thru 0xda4bae5 (0x7bf0 bytes) 2024-09-21T15:34:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:4717 write 0xb4d3dd0 thru 0xb4dcf6f (0x91a0 bytes) 2024-09-21T15:34:24.383 INFO:teuthology.orchestra.run.smithi107.stdout:4718 read 0xb14b4f2 thru 0xb14cd9c (0x18ab bytes) 2024-09-21T15:34:24.383 INFO:teuthology.orchestra.run.smithi107.stdout:4720 read 0x95387ba thru 0x9545f0f (0xd756 bytes) 2024-09-21T15:34:24.383 INFO:teuthology.orchestra.run.smithi107.stdout:4721 read 0x534fd22 thru 0x535f1da (0xf4b9 bytes) 2024-09-21T15:34:24.383 INFO:teuthology.orchestra.run.smithi107.stdout:4723 read 0x20255f6 thru 0x202ae65 (0x5870 bytes) 2024-09-21T15:34:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:4724 read 0x76442e5 thru 0x764a2a2 (0x5fbe bytes) 2024-09-21T15:34:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:4725 read 0x146aa28 thru 0x1470ad5 (0x60ae bytes) 2024-09-21T15:34:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:4726 punch from 0xb3f5b00 to 0xb3fcb7f, (0x707f bytes) 2024-09-21T15:34:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:4727 writesame 0x4b9068c thru 0x4b9a5b2 data_size 0x1b(0x9f27 bytes) 2024-09-21T15:34:24.387 INFO:teuthology.orchestra.run.smithi107.stdout:4729 writesame 0x5ec032e thru 0x5ec8d3a data_size 0x1(0x8a0d bytes) 2024-09-21T15:34:24.411 INFO:teuthology.orchestra.run.smithi107.stdout:4730 write 0x7c8b005 thru 0x7c94bce (0x9bca bytes) 2024-09-21T15:34:24.412 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:24.408+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:24.422 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.422 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.425 INFO:teuthology.orchestra.run.smithi107.stdout:4731 write 0xdc25d89 thru 0xdc2f513 (0x978b bytes) 2024-09-21T15:34:24.430 INFO:teuthology.orchestra.run.smithi107.stdout:4732 write 0x17eafb0 thru 0x17f226b (0x72bc bytes) 2024-09-21T15:34:24.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:24.443 INFO:teuthology.orchestra.run.smithi107.stdout:4733 write 0xe09275c thru 0xe09dddd (0xb682 bytes) 2024-09-21T15:34:24.448 INFO:teuthology.orchestra.run.smithi107.stdout:4734 writesame 0x1dc0e40 thru 0x1dc1fac data_size 0x1(0x116d bytes) 2024-09-21T15:34:24.451 INFO:teuthology.orchestra.run.smithi107.stdout:4735 punch from 0x10c3476 to 0x10d1065, (0xdbef bytes) 2024-09-21T15:34:24.451 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.451 INFO:teuthology.orchestra.run.smithi107.stdout:4738 write 0x917dce5 thru 0x918d321 (0xf63d bytes) 2024-09-21T15:34:24.457 INFO:teuthology.orchestra.run.smithi107.stdout:4739 write 0xb4fa0d7 thru 0xb503eb2 (0x9ddc bytes) 2024-09-21T15:34:24.462 INFO:teuthology.orchestra.run.smithi107.stdout:4740 writesame 0x163d3c3 thru 0x16460dc data_size 0x1(0x8d1a bytes) 2024-09-21T15:34:24.482 INFO:teuthology.orchestra.run.smithi107.stdout:4741 punch from 0x3e5043b to 0x3e53b3b, (0x3700 bytes) 2024-09-21T15:34:24.482 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.483 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.483 INFO:teuthology.orchestra.run.smithi107.stdout:4745 writesame 0xfca29c thru 0xfcd952 data_size 0x45(0x36b7 bytes) 2024-09-21T15:34:24.496 INFO:teuthology.orchestra.run.smithi107.stdout:4747 trunc from 0xe09ddde to 0x6af4d8e 2024-09-21T15:34:24.510 INFO:teuthology.orchestra.run.smithi107.stdout:4749 write 0x3a97d22 thru 0x3a99aed (0x1dcc bytes) 2024-09-21T15:34:24.512 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:24.510+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:24.515 INFO:teuthology.orchestra.run.smithi107.stdout:4751 read 0x54bc057 thru 0x54bed88 (0x2d32 bytes) 2024-09-21T15:34:24.515 INFO:teuthology.orchestra.run.smithi107.stdout:4752 punch from 0x33a6c7b to 0x33aaf18, (0x429d bytes) 2024-09-21T15:34:24.515 INFO:teuthology.orchestra.run.smithi107.stdout:4753 write 0xd6f20d thru 0xd73af3 (0x48e7 bytes) 2024-09-21T15:34:24.537 INFO:teuthology.orchestra.run.smithi107.stdout:4754 write 0xe3617df thru 0xe3676fb (0x5f1d bytes) 2024-09-21T15:34:24.541 INFO:teuthology.orchestra.run.smithi107.stdout:4755 read 0x14c2a07 thru 0x14c7fe4 (0x55de bytes) 2024-09-21T15:34:24.542 INFO:teuthology.orchestra.run.smithi107.stdout:4758 read 0xd977a9a thru 0xd97e12e (0x6695 bytes) 2024-09-21T15:34:24.542 INFO:teuthology.orchestra.run.smithi107.stdout:4759 writesame 0x6f44cc0 thru 0x6f47d81 data_size 0x2(0x30c2 bytes) 2024-09-21T15:34:24.548 INFO:teuthology.orchestra.run.smithi107.stdout:4760 write 0xd2888df thru 0xd28c97d (0x409f bytes) 2024-09-21T15:34:24.552 INFO:teuthology.orchestra.run.smithi107.stdout:4763 punch from 0x1fa3dbf to 0x1fb2691, (0xe8d2 bytes) 2024-09-21T15:34:24.552 INFO:teuthology.orchestra.run.smithi107.stdout:4764 write 0x4e503ea thru 0x4e50506 (0x11d bytes) 2024-09-21T15:34:24.556 INFO:teuthology.orchestra.run.smithi107.stdout:4765 write 0xc58e388 thru 0xc59ba69 (0xd6e2 bytes) 2024-09-21T15:34:24.563 INFO:teuthology.orchestra.run.smithi107.stdout:4766 write 0x6391bf1 thru 0x6399577 (0x7987 bytes) 2024-09-21T15:34:24.565 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.565 INFO:teuthology.orchestra.run.smithi107.stdout:4768 write 0xa55cb48 thru 0xa563622 (0x6adb bytes) 2024-09-21T15:34:24.570 INFO:teuthology.orchestra.run.smithi107.stdout:4769 punch from 0xa5012cb to 0xa5109ba, (0xf6ef bytes) 2024-09-21T15:34:24.570 INFO:teuthology.orchestra.run.smithi107.stdout:4771 read 0xe3328a4 thru 0xe34096c (0xe0c9 bytes) 2024-09-21T15:34:24.570 INFO:teuthology.orchestra.run.smithi107.stdout:4773 writesame 0x3054a36 thru 0x30628d8 data_size 0x1(0xdea3 bytes) 2024-09-21T15:34:24.613 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:24.612+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:24.618 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.618 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.626 INFO:teuthology.orchestra.run.smithi107.stdout:4775 trunc from 0xe3676fc to 0x1bc8e63 2024-09-21T15:34:24.632 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:24.674 INFO:teuthology.orchestra.run.smithi107.stdout:4778 write 0xc52de59 thru 0xc5390a4 (0xb24c bytes) 2024-09-21T15:34:24.680 INFO:teuthology.orchestra.run.smithi107.stdout:4779 punch from 0x5bf7ecf to 0x5c001a8, (0x82d9 bytes) 2024-09-21T15:34:24.689 INFO:teuthology.orchestra.run.smithi107.stdout:4780 write 0xe59f7d5 thru 0xe5a7de9 (0x8615 bytes) 2024-09-21T15:34:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:4782 read 0x8688967 thru 0x8690d38 (0x83d2 bytes) 2024-09-21T15:34:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:4783 write 0xa2f7ca4 thru 0xa2ff36c (0x76c9 bytes) 2024-09-21T15:34:24.698 INFO:teuthology.orchestra.run.smithi107.stdout:4784 write 0x6324b3 thru 0x633b02 (0x1650 bytes) 2024-09-21T15:34:24.703 INFO:teuthology.orchestra.run.smithi107.stdout:4786 punch from 0xcdf9067 to 0xcdfa42c, (0x13c5 bytes) 2024-09-21T15:34:24.711 INFO:teuthology.orchestra.run.smithi107.stdout:4787 trunc from 0xe5a7dea to 0x9a561cc 2024-09-21T15:34:24.714 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:24.712+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:24.742 INFO:teuthology.orchestra.run.smithi107.stdout:4788 write 0xb77d149 thru 0xb782409 (0x52c1 bytes) 2024-09-21T15:34:24.766 INFO:teuthology.orchestra.run.smithi107.stdout:4789 trunc from 0xb78240a to 0x8dc364 2024-09-21T15:34:24.791 INFO:teuthology.orchestra.run.smithi107.stdout:4790 read 0x2d9bb3 thru 0x2dd6c8 (0x3b16 bytes) 2024-09-21T15:34:24.792 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.792 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.792 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.792 INFO:teuthology.orchestra.run.smithi107.stdout:4792 read 0x781313 thru 0x78f160 (0xde4e bytes) 2024-09-21T15:34:24.808 INFO:teuthology.orchestra.run.smithi107.stdout:4793 writesame 0x4ba3276 thru 0x4baabb2 data_size 0x1(0x793d bytes) 2024-09-21T15:34:24.812 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:24.828 INFO:teuthology.orchestra.run.smithi107.stdout:4794 writesame 0xbf10641 thru 0xbf1a18d data_size 0x1(0x9b4d bytes) 2024-09-21T15:34:24.829 INFO:teuthology.orchestra.run.smithi107.stdout:4795 writesame 0xbd8b495 thru 0xbd9869a data_size 0x3a(0xd206 bytes) 2024-09-21T15:34:24.829 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:24.826+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:24.839 INFO:teuthology.orchestra.run.smithi107.stdout:4796 writesame 0x2025572 thru 0x202707f data_size 0x1(0x1b0e bytes) 2024-09-21T15:34:24.859 INFO:teuthology.orchestra.run.smithi107.stdout:4797 trunc from 0xbf1a18e to 0x4674652 2024-09-21T15:34:24.877 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.877 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:24.884 INFO:teuthology.orchestra.run.smithi107.stdout:4803 trunc from 0x4674652 to 0xbb9ed 2024-09-21T15:34:24.907 INFO:teuthology.orchestra.run.smithi107.stdout:4804 punch from 0x33c28 to 0x39a3a, (0x5e12 bytes) 2024-09-21T15:34:24.907 INFO:teuthology.orchestra.run.smithi107.stdout:4806 read 0x2d9b6 thru 0x3c051 (0xe69c bytes) 2024-09-21T15:34:24.916 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:24.914+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:24.925 INFO:teuthology.orchestra.run.smithi107.stdout:4811 write 0x6cc2232 thru 0x6cc8813 (0x65e2 bytes) 2024-09-21T15:34:24.931 INFO:teuthology.orchestra.run.smithi107.stdout:4813 write 0x2bf8527 thru 0x2bf8922 (0x3fc bytes) 2024-09-21T15:34:24.938 INFO:teuthology.orchestra.run.smithi107.stdout:4814 read 0x4fb164b thru 0x4fb6e6b (0x5821 bytes) 2024-09-21T15:34:24.938 INFO:teuthology.orchestra.run.smithi107.stdout:4815 writesame 0x33a0bad thru 0x33ab4e8 data_size 0x2(0xa93c bytes) 2024-09-21T15:34:24.956 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:24.956 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:24.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:24.974 INFO:teuthology.orchestra.run.smithi107.stdout:4817 read 0x14c370b thru 0x14cbbba (0x84b0 bytes) 2024-09-21T15:34:24.974 INFO:teuthology.orchestra.run.smithi107.stdout:4821 read 0x2914e91 thru 0x2919eed (0x505d bytes) 2024-09-21T15:34:24.977 INFO:teuthology.orchestra.run.smithi107.stdout:4822 trunc from 0x6cc8814 to 0x48ec746 2024-09-21T15:34:24.993 INFO:teuthology.orchestra.run.smithi107.stdout:4823 read 0x10f87ce thru 0x11044b3 (0xbce6 bytes) 2024-09-21T15:34:25.007 INFO:teuthology.orchestra.run.smithi107.stdout:4824 trunc from 0x48ec746 to 0xbcd5e3b 2024-09-21T15:34:25.017 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:25.016+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:25.018 INFO:teuthology.orchestra.run.smithi107.stdout:4825 trunc from 0xbcd5e3b to 0xd3212b1 2024-09-21T15:34:25.044 INFO:teuthology.orchestra.run.smithi107.stdout:4826 trunc from 0xd3212b1 to 0x308b41d 2024-09-21T15:34:25.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:25.071 INFO:teuthology.orchestra.run.smithi107.stdout:4828 writesame 0x3d2ac9c thru 0x3d376a6 data_size 0x15(0xca0b bytes) 2024-09-21T15:34:25.072 INFO:teuthology.orchestra.run.smithi107.stdout:4829 punch from 0x347719 to 0x357182, (0xfa69 bytes) 2024-09-21T15:34:25.072 INFO:teuthology.orchestra.run.smithi107.stdout:4832 punch from 0x3b3d73c to 0x3b3f1c3, (0x1a87 bytes) 2024-09-21T15:34:25.072 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:25.072 INFO:teuthology.orchestra.run.smithi107.stdout:4835 clone 35 order 25 su 4194304 sc 11 2024-09-21T15:34:25.118 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:25.118 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:25.118 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:25.116+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:25.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:25.164 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:34:25.166 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T15:34:25.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T15:34:25.236 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:25.236 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:25.236 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:25.234+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:25.266 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone34 from 0x3d376a7 (overlap 0xbb9ed) to 0xa8dbb 2024-09-21T15:34:25.270 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:25.270 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:25.300 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:25.319 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:25.318+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:25.331 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:25.331 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:25.336 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:25.352 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:25.352 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:25.353 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:25.372 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T15:34:25.377 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:25.377 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:25.398 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:25.398 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:25.419 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:25.419 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:25.420 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:25.418+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:25.519 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:25.519 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:25.519 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:25.519 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:25.519 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:25.522 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:25.520+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:25.534 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:25.622 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:25.618+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:25.660 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.536677458, 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:25.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:25.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:25.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.094521245000003, 2024-09-21T15:34:25.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:25.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:25.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.094496499000002, 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:25.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:25.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.094451548999999, 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090594891000002, 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090550530999998, 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.687 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090497182, 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090465789, 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.695 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090420063000003, 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:25.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:25.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.09039069, 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:25.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:25.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:25.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.09036613, 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:25.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.090339493000002, 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086500925999999, 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:25.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086471105999998, 2024-09-21T15:34:25.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:25.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086445576000003, 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.720 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.08641987, 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086369830000002, 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:25.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:25.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086335030000001, 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:25.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:25.735 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.086310380999997, 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.736 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.739 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 46.082259974000003, 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.740 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:25.741 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:25.742 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:25.743 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:25.744 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:25.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:25.745 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:25.742+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:25.821 INFO:teuthology.orchestra.run.smithi107.stdout:4836 trunc from 0x3d376a7 to 0x302817e 2024-09-21T15:34:25.845 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:25.844+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:25.848 INFO:teuthology.orchestra.run.smithi107.stdout:4839 write 0x913e4b thru 0x920ea4 (0xd05a bytes) 2024-09-21T15:34:25.871 INFO:teuthology.orchestra.run.smithi107.stdout:4841 writesame 0x88f5d97 thru 0x89010e1 data_size 0x7(0xb34b bytes) 2024-09-21T15:34:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:4843 read 0x318c6ee thru 0x318d995 (0x12a8 bytes) 2024-09-21T15:34:25.891 INFO:teuthology.orchestra.run.smithi107.stdout:4844 write 0xd610b02 thru 0xd611a00 (0xeff bytes) 2024-09-21T15:34:25.896 INFO:teuthology.orchestra.run.smithi107.stdout:4845 write 0x8b36151 thru 0x8b3f34a (0x91fa bytes) 2024-09-21T15:34:25.898 INFO:teuthology.orchestra.run.smithi107.stdout:4846 write 0xd43b3e8 thru 0xd44111a (0x5d33 bytes) 2024-09-21T15:34:25.909 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:25.909 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:25.909 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:25.909 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:25.910 INFO:teuthology.orchestra.run.smithi107.stdout:4847 trunc from 0xd611a01 to 0x65247ce 2024-09-21T15:34:25.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:25.940 INFO:teuthology.orchestra.run.smithi107.stdout:4848 read 0x603576b thru 0x603fc72 (0xa508 bytes) 2024-09-21T15:34:25.941 INFO:teuthology.orchestra.run.smithi107.stdout:4850 read 0x450979a thru 0x450ddcb (0x4632 bytes) 2024-09-21T15:34:25.946 INFO:teuthology.orchestra.run.smithi107.stdout:4851 trunc from 0x65247ce to 0x909d28b 2024-09-21T15:34:25.946 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:25.942+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:25.965 INFO:teuthology.orchestra.run.smithi107.stdout:4852 trunc from 0x909d28b to 0x32b1fe3 2024-09-21T15:34:26.006 INFO:teuthology.orchestra.run.smithi107.stdout:4853 write 0xe8cfcaa thru 0xe8d422d (0x4584 bytes) 2024-09-21T15:34:26.008 INFO:teuthology.orchestra.run.smithi107.stdout:4854 read 0x9ad93d8 thru 0x9ae5a73 (0xc69c bytes) 2024-09-21T15:34:26.018 INFO:teuthology.orchestra.run.smithi107.stdout:4855 trunc from 0xe8d422e to 0x637b3b4 2024-09-21T15:34:26.037 INFO:teuthology.orchestra.run.smithi107.stdout:4856 punch from 0x4d67628 to 0x4d6dbb9, (0x6591 bytes) 2024-09-21T15:34:26.037 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:26.037 INFO:teuthology.orchestra.run.smithi107.stdout:4858 read 0x19464f9 thru 0x194dd1c (0x7824 bytes) 2024-09-21T15:34:26.047 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:26.042+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:4859 write 0xcc1df14 thru 0xcc2244f (0x453c bytes) 2024-09-21T15:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:4860 trunc from 0xcc22450 to 0xda73d69 2024-09-21T15:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:26.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:4861 trunc from 0xda73d69 to 0x459cccd 2024-09-21T15:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:4862 read 0x180388d thru 0x180d352 (0x9ac6 bytes) 2024-09-21T15:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:4865 write 0xa667467 thru 0xa66cf5d (0x5af7 bytes) 2024-09-21T15:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:4867 punch from 0x8f3b431 to 0x8f49fde, (0xebad bytes) 2024-09-21T15:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:4869 read 0x409951f thru 0x40a5740 (0xc222 bytes) 2024-09-21T15:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:4870 write 0x149d0f8 thru 0x14a0312 (0x321b bytes) 2024-09-21T15:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:4872 read 0x2659647 thru 0x265ac2b (0x15e5 bytes) 2024-09-21T15:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:4873 read 0x150e2d5 thru 0x15144a0 (0x61cc bytes) 2024-09-21T15:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:4874 read 0x5a398a5 thru 0x5a47ad6 (0xe232 bytes) 2024-09-21T15:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:4875 read 0x6b6ea0d thru 0x6b7972a (0xad1e bytes) 2024-09-21T15:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:4876 trunc from 0xa66cf5e to 0xadf78d8 2024-09-21T15:34:26.148 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:26.146+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:4878 trunc from 0xadf78d8 to 0x63f1bb4 2024-09-21T15:34:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T15:34:26.199 INFO:teuthology.orchestra.run.smithi107.stdout:4879 writesame 0xb8cc354 thru 0xb8d1712 data_size 0x1(0x53bf bytes) 2024-09-21T15:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout:4881 punch from 0x7f5ee59 to 0x7f6ec38, (0xfddf bytes) 2024-09-21T15:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout:4882 punch from 0x6265793 to 0x6265b84, (0x3f1 bytes) 2024-09-21T15:34:26.202 INFO:teuthology.orchestra.run.smithi107.stdout:4883 read 0xa99c9b0 thru 0xa9a3d28 (0x7379 bytes) 2024-09-21T15:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout:4885 punch from 0x1b1331 to 0x1c0810, (0xf4df bytes) 2024-09-21T15:34:26.203 INFO:teuthology.orchestra.run.smithi107.stdout:4886 write 0xb0b09 thru 0xb2020 (0x1518 bytes) 2024-09-21T15:34:26.205 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T15:34:26.218 INFO:teuthology.orchestra.run.smithi107.stdout:4887 write 0xc65e07a thru 0xc65f327 (0x12ae bytes) 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.221 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 89.173094324999994, 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.122781118999995, 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 79.111611210000007, 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 17.696049913, 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154140 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'505, mlcod=88'500)", 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.2178439479999996, 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061868679999999999, 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002765+0000", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002772+0000", 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1799999999999999e-06 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002785+0000", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2716e-05 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002795+0000", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7510000000000003e-06 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002835+0000", 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0129000000000002e-05 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002855+0000", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9741e-05 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008875+0000", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060205340000000001 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008953+0000", 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7595999999999994e-05 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.241 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154141 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'506, mlcod=88'501)", 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.2160706709999998, 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044846160000000003, 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004538+0000", 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004545+0000", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999998e-06 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004558+0000", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2847000000000001e-05 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004565+0000", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0280000000000003e-06 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004600+0000", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4774999999999998e-05 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004612+0000", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2398000000000001e-05 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008961+0000", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043488329999999999 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.009024+0000", 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2484999999999996e-05 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154143 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'507, mlcod=88'501)", 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.2148718230000002, 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053764679999999997, 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005737+0000", 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005744+0000", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0210000000000002e-06 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005758+0000", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005765+0000", 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5180000000000001e-06 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005854+0000", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9349000000000001e-05 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005873+0000", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814e-05 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011030+0000", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051572900000000001 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011114+0000", 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4271999999999995e-05 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154146 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'509, mlcod=88'503)", 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 5.2124395750000003, 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055853480000000004, 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008169+0000", 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008177+0000", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8790000000000003e-06 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008190+0000", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2362e-05 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008198+0000", 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1340000000000004e-06 2024-09-21T15:34:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008440+0000", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024192699999999999 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008459+0000", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9655000000000001e-05 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013713+0000", 2024-09-21T15:34:26.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052535560000000004 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013756+0000", 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2608999999999998e-05 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.9051681049999996, 2024-09-21T15:34:26.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:34:26.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:34:26.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:34:26.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T15:34:26.260 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.5281560499999998, 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056653390000000001, 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4204", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.170:0/3270855337", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692450+0000", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692472+0000", 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1977e-05 2024-09-21T15:34:26.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692511+0000", 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8708999999999998e-05 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692528+0000", 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7640000000000001e-05 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.697842+0000", 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053131849999999998 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.698119+0000", 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027754100000000002 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157210 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.390343524, 2024-09-21T15:34:26.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005552487, 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157210 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830265+0000", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830272+0000", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.708e-06 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830287+0000", 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5341000000000001e-05 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830295+0000", 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5129999999999999e-06 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830345+0000", 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9704999999999999e-05 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830427+0000", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2451000000000001e-05 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830626+0000", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019875599999999999 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832481+0000", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001855506 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835763+0000", 2024-09-21T15:34:26.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032813460000000001 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835790+0000", 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7793e-05 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835819+0000", 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8416e-05 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.269 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157211 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.39028648100000002, 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056339160000000001, 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157211 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830322+0000", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830328+0000", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5969999999999999e-06 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830339+0000", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029000000000001e-05 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830347+0000", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830948+0000", 2024-09-21T15:34:26.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00060113699999999996 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831061+0000", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112988 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831448+0000", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038673899999999998 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832485+0000", 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001037145 2024-09-21T15:34:26.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835875+0000", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033895459999999998 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835922+0000", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7061000000000002e-05 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835957+0000", 2024-09-21T15:34:26.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5478000000000002e-05 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157212 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.39024857600000001, 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056857399999999999, 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157212 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830366+0000", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.485e-06 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830376+0000", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8619999999999998e-06 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830383+0000", 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831965+0000", 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015819709999999999 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832013+0000", 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8306999999999997e-05 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832208+0000", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000194928 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.833459+0000", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012513240000000001 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835987+0000", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002528069 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836013+0000", 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5677000000000002e-05 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836047+0000", 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4103999999999997e-05 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157214 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.38897290200000001, 2024-09-21T15:34:26.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049247070000000004, 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157214 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831636+0000", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831642+0000", 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.533e-06 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831655+0000", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2724999999999999e-05 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831662+0000", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5919999999999997e-06 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832518+0000", 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00085589900000000003 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832605+0000", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7318000000000007e-05 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832974+0000", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036939200000000002 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.833818+0000", 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000843799 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836480+0000", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026621840000000002 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836530+0000", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9721e-05 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836562+0000", 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1528e-05 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157272 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.307038221, 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006546451, 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157272 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913569+0000", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913584+0000", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5364000000000001e-05 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913628+0000", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4175999999999999e-05 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913655+0000", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6452999999999999e-05 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913731+0000", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6252000000000001e-05 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913823+0000", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2430999999999996e-05 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914239+0000", 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041542199999999998 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918388+0000", 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041490950000000002 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920017+0000", 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001628839 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920066+0000", 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9453000000000002e-05 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920118+0000", 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157273 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.291 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30691664499999999, 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0067711760000000003, 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157273 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913691+0000", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913702+0000", 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0305999999999999e-05 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913722+0000", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0013e-05 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913738+0000", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6033e-05 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914805+0000", 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001067248 2024-09-21T15:34:26.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.915000+0000", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000194744 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.915803+0000", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00080309000000000001 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918394+0000", 2024-09-21T15:34:26.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025913780000000001 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920224+0000", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001830103 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920325+0000", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101229 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920464+0000", 2024-09-21T15:34:26.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013900199999999999 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157274 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30685802000000001, 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006850068, 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.297 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157274 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913751+0000", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913758+0000", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6399999999999997e-06 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913776+0000", 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7677e-05 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913789+0000", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2643000000000001e-05 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.916645+0000", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028568629999999999 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.916736+0000", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0342999999999999e-05 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917114+0000", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037831099999999999 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920134+0000", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030198690000000001 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920524+0000", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038972099999999999 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920569+0000", 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5352000000000002e-05 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920602+0000", 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2781999999999999e-05 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157275 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:26.302 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30680794300000003, 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070002429999999997, 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157275 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913801+0000", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913807+0000", 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5300000000000002e-06 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913824+0000", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6444e-05 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913835+0000", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1636e-05 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917625+0000", 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037894550000000002 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917693+0000", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8073999999999999e-05 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917990+0000", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029714900000000002 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920616+0000", 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026256560000000001 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920670+0000", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3885999999999999e-05 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920746+0000", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.648e-05 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920802+0000", 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5940000000000003e-05 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.307 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157276 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30606655999999999, 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073072780000000004, 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157276 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.308 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914541+0000", 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914575+0000", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4075999999999997e-05 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914603+0000", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.747e-05 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914617+0000", 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4216000000000001e-05 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918460+0000", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038429660000000002 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918588+0000", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128362 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.919111+0000", 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052255200000000002 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921667+0000", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025559570000000002 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921723+0000", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6477999999999997e-05 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921792+0000", 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9283999999999996e-05 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921851+0000", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.821e-05 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157277 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 0.30570514799999998, 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0091388249999999997, 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.313 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157277 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914903+0000", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914912+0000", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6449999999999994e-06 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914928+0000", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6413999999999999e-05 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914939+0000", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0864e-05 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920836+0000", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058964730000000002 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920922+0000", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6340000000000003e-05 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921264+0000", 2024-09-21T15:34:26.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034240399999999998 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.922131+0000", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086667199999999997 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.923949+0000", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018179649999999999 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.924005+0000", 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6431000000000002e-05 2024-09-21T15:34:26.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.924043+0000", 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8009999999999997e-05 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.318 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:26.320 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:26.320 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:26.314+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:26.421 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:26.420+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:26.475 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:26.476 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:26.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:26.532 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:26.530+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:26.600 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:26.613 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:26.623 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:26.620+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:26.723 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:26.720+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:26.742 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T15:34:26.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 94.426994969999996, 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:26.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:26.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:26.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:26.802 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 94.398783793999996, 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:26.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:26.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.209484426000003, 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.078099734999995, 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:26.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:26.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 89.917002331000006, 2024-09-21T15:34:26.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:26.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 89.916949783999996, 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:26.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 89.916901659000004, 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:26.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 89.904504771000006, 2024-09-21T15:34:26.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:26.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:26.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.840 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.343731789000003, 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.841 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:26.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:26.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.276707423000005, 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:26.852 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.141587291999997, 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:26.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:26.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.100925149000005, 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.858 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:26.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:26.862 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.100911932000002, 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.866 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.100899124999998, 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:26.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.870 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.100858717999998, 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:26.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.874 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.100834250000005, 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.878 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.081612261999993, 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 87.994308679, 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:26.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.999623236000001, 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:26.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:26.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:26.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.895 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 19.585363889, 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:26.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:26.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:26.902 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:26.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.904 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:26.905 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:26.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:26.908 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:26.902+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:27.008 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:27.006+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:27.035 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:27.035 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:27.035 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:27.035 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:27.049 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:27.108 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:27.104+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:27.171 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:27.171 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:27.171 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:27.171 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:27.171 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:27.185 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:27.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:34:27.240 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:27.238+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:27.310 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:27.306+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.07061092, 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.350 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:27.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070586860999999, 2024-09-21T15:34:27.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:27.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070562897999999, 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:27.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.362 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070539132, 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.366 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070515489999998, 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:27.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:27.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070489937000001, 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.371 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070445721999999, 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:27.375 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.376 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:27.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.379 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070417013, 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:27.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.383 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070385470000002, 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070358398, 2024-09-21T15:34:27.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:27.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.392 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.070332465, 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:27.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:27.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.069797522000002, 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:27.397 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.069167854, 2024-09-21T15:34:27.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:27.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:27.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.405 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.065256291000001, 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:27.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.409 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.065209793000001, 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:27.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:27.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:27.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:27.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.065160624000001, 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:27.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.065112775999999, 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:27.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:27.421 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.065064387, 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:27.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.064983466000001, 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.426 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:27.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 20.061385102999999, 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:27.430 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:27.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:27.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:27.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:27.434 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:27.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:27.436 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:27.430+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:27.442 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:27.536 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:27.534+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:27.561 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:27.562 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:27.562 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:27.577 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:27.581 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:27.581 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:27.597 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:27.597 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:27.616 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:27.616 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:27.632 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:27.632 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:27.637 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:27.634+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:27.649 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:27.649 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:27.667 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:27.685 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:27.685 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:27.697 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:27.697 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:27.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:27.705 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:27.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:27.743 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T15:34:27.744 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:34:27.757 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:27.756+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:27.762 INFO:teuthology.orchestra.run.smithi107.stdout:4888 writesame 0x7e39030 thru 0x7e46a00 data_size 0x1(0xd9d1 bytes) 2024-09-21T15:34:27.778 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T15:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:27.802 INFO:teuthology.orchestra.run.smithi107.stdout:4891 write 0xaf78b6a thru 0xaf8876b (0xfc02 bytes) 2024-09-21T15:34:27.816 INFO:teuthology.orchestra.run.smithi107.stdout:4892 write 0xc6cfd00 thru 0xc6df1e4 (0xf4e5 bytes) 2024-09-21T15:34:27.818 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:27.827 INFO:teuthology.orchestra.run.smithi107.stdout:4894 trunc from 0xc6df1e5 to 0x7920536 2024-09-21T15:34:27.839 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:27.834+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:27.847 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:27.847 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:27.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:27.949 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:27.948+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout:4897 read 0x5e1a02 thru 0x5e95ba (0x7bb9 bytes) 2024-09-21T15:34:27.970 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:27.987 INFO:teuthology.orchestra.run.smithi107.stdout:4899 writesame 0xd404165 thru 0xd405514 data_size 0x18(0x13b0 bytes) 2024-09-21T15:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout:4900 read 0xa811022 thru 0xa8127c0 (0x179f bytes) 2024-09-21T15:34:27.989 INFO:teuthology.orchestra.run.smithi107.stdout:4901 read 0x9c80411 thru 0x9c8145a (0x104a bytes) 2024-09-21T15:34:28.007 INFO:teuthology.orchestra.run.smithi107.stdout:4902 trunc from 0xd405515 to 0x1f15f8f 2024-09-21T15:34:28.028 INFO:teuthology.orchestra.run.smithi107.stdout:4903 write 0x17d830f thru 0x17e6dad (0xea9f bytes) 2024-09-21T15:34:28.041 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:28.038+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:28.042 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.042 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.042 INFO:teuthology.orchestra.run.smithi107.stdout:4905 write 0x358f07f thru 0x3593adc (0x4a5e bytes) 2024-09-21T15:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:28.045 INFO:teuthology.orchestra.run.smithi107.stdout:4907 punch from 0x348840e to 0x3490e43, (0x8a35 bytes) 2024-09-21T15:34:28.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:28.069 INFO:teuthology.orchestra.run.smithi107.stdout:4908 writesame 0xe5578d7 thru 0xe55fa6f data_size 0x1(0x8199 bytes) 2024-09-21T15:34:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:4909 clone 36 order 20 su 16384 sc 10 2024-09-21T15:34:28.145 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:28.144+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:28.225 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.226 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:28.281 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:28.280+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:28.344 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:28.342+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:28.403 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.403 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.417 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:28.445 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:28.442+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:28.536 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.536 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.545 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:28.542+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:28.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:28.647 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:28.642+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:28.671 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.671 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:28.748 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T15:34:28.750 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:28.746+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:28.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T15:34:28.814 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.814 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:28.848 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:28.844+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:28.949 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:28.948+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:28.980 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:28.980 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:28.995 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:29.050 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:29.048+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:29.110 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.110 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.123 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:29.151 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:29.148+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:29.246 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.247 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.252 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:29.252+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:29.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:29.353 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:29.350+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:29.454 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:29.452+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:29.552 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.552 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.555 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:29.552+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:29.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:29.656 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:29.654+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:29.688 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.689 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:29.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:29.766 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T15:34:29.766 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:29.762+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:29.772 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:34:29.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T15:34:29.811 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone35 from 0xe55fa70 (overlap 0x1f15f8f) to 0x953644 2024-09-21T15:34:29.858 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:29.854+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:29.890 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:29.891 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:29.904 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:29.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:29.927 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:29.946 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:29.946 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:29.958 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:29.954+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:29.963 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:29.964 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:29.984 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:29.984 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:30.005 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:30.005 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:30.024 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:30.024 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:30.044 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:30.044 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:30.051 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:30.051 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:30.051 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:30.059 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:30.056+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:30.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:30.160 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:30.156+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:3381 3.9 e30/21 3:91379144:::rbd_object_map.1191cae494be:head v 30'22, mlcod=30'19)", 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:30.223 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.172823625000007, 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056170370000000001, 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047515+0000", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047533+0000", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7887999999999999e-05 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047544+0000", 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1566000000000001e-05 2024-09-21T15:34:30.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047557+0000", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2136999999999999e-05 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047599+0000", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2778999999999998e-05 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.047621+0000", 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1219000000000001e-05 2024-09-21T15:34:30.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053092+0000", 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0054711889999999996 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:57.053133+0000", 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0766000000000002e-05 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.227 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7322 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'203, mlcod=36'199)", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.122510418999994, 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051022890000000003, 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097829+0000", 2024-09-21T15:34:30.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097828+0000", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097835+0000", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4399999999999999e-06 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097845+0000", 2024-09-21T15:34:30.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0057e-05 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097852+0000", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.365e-06 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097885+0000", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2962000000000001e-05 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.097897+0000", 2024-09-21T15:34:30.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2174000000000001e-05 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102906+0000", 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005009416 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.102931+0000", 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4623000000000001e-05 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:7330 3.0 e36/21 3:04220828:::rbd_object_map.1191c706ae32:head v 36'207, mlcod=36'204)", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 83.111340510000005, 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0047166939999999996, 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108999+0000", 2024-09-21T15:34:30.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.108998+0000", 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109006+0000", 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9060000000000002e-06 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109020+0000", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4576000000000001e-05 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109027+0000", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.2030000000000003e-06 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109068+0000", 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1017999999999999e-05 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.109089+0000", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0492e-05 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113690+0000", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.004600982 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:07.113715+0000", 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5482000000000001e-05 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.235 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:124630 3.0 e80/50 3:0430ea1e:::rbd_data.119163e0d152.0000000000000003:head v 80'444, mlcod=79'442)", 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 21.695779213000002, 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016535583999999999, 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524560+0000", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.524559+0000", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528885+0000", 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043258159999999997 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528912+0000", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6254e-05 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528922+0000", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0455000000000001e-05 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.528992+0000", 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9678999999999995e-05 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.529012+0000", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0846e-05 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541013+0000", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012000062000000001 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:08.541095+0000", 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2935000000000005e-05 2024-09-21T15:34:30.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154140 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'505, mlcod=88'500)", 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2175732480000008, 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061868679999999999, 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:30.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002766+0000", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002765+0000", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002772+0000", 2024-09-21T15:34:30.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1799999999999999e-06 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002785+0000", 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2716e-05 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002795+0000", 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7510000000000003e-06 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002835+0000", 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0129000000000002e-05 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.002855+0000", 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9741e-05 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008875+0000", 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0060205340000000001 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008953+0000", 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7595999999999994e-05 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154141 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'506, mlcod=88'501)", 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2157999709999991, 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0044846160000000003, 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004539+0000", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004538+0000", 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004545+0000", 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9369999999999998e-06 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004558+0000", 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2847000000000001e-05 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004565+0000", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0280000000000003e-06 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004600+0000", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4774999999999998e-05 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.004612+0000", 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2398000000000001e-05 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008961+0000", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0043488329999999999 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.009024+0000", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2484999999999996e-05 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154143 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'507, mlcod=88'501)", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:30.248 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2146011229999996, 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053764679999999997, 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005738+0000", 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005737+0000", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005744+0000", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0210000000000002e-06 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005758+0000", 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005765+0000", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5180000000000001e-06 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005854+0000", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9349000000000001e-05 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.005873+0000", 2024-09-21T15:34:30.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8814e-05 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011030+0000", 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0051572900000000001 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.011114+0000", 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4271999999999995e-05 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:154146 3.0 e88/50 3:06f90ff3:::rbd_object_map.11915dac012e:head v 88'509, mlcod=88'503)", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 9.2121688749999997, 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0055853480000000004, 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008170+0000", 2024-09-21T15:34:30.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008169+0000", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008177+0000", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8790000000000003e-06 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008190+0000", 2024-09-21T15:34:30.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2362e-05 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008198+0000", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1340000000000004e-06 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008440+0000", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00024192699999999999 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.008459+0000", 2024-09-21T15:34:30.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9655000000000001e-05 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013713+0000", 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052535560000000004 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.013756+0000", 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2608999999999998e-05 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.9048974049999998, 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:30.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:34:30.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 8.52788535, 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056653390000000001, 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "reached pg", 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4204", 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.170:0/3270855337", 2024-09-21T15:34:30.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 5 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692454+0000", 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692450+0000", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692472+0000", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1977e-05 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692511+0000", 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8708999999999998e-05 2024-09-21T15:34:30.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.692528+0000", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7640000000000001e-05 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.697842+0000", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0053131849999999998 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.698119+0000", 2024-09-21T15:34:30.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00027754100000000002 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157210 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3900728239999998, 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.005552487, 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157210 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830266+0000", 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830265+0000", 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830272+0000", 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.708e-06 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830287+0000", 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5341000000000001e-05 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830295+0000", 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5129999999999999e-06 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830345+0000", 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9704999999999999e-05 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830427+0000", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2451000000000001e-05 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830626+0000", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00019875599999999999 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832481+0000", 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001855506 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835763+0000", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0032813460000000001 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835790+0000", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7793e-05 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835819+0000", 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8416e-05 2024-09-21T15:34:30.272 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157211 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3900157809999998, 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056339160000000001, 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157211 2024-09-21T15:34:30.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830323+0000", 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830322+0000", 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830328+0000", 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5969999999999999e-06 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830339+0000", 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029000000000001e-05 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830347+0000", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0409999999999998e-06 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830948+0000", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00060113699999999996 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831061+0000", 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112988 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831448+0000", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038673899999999998 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832485+0000", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001037145 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835875+0000", 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0033895459999999998 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835922+0000", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7061000000000002e-05 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835957+0000", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5478000000000002e-05 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157212 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:30.278 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3899778759999997, 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0056857399999999999, 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157212 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830361+0000", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830366+0000", 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.485e-06 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830376+0000", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8619999999999998e-06 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.830383+0000", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831965+0000", 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015819709999999999 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832013+0000", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8306999999999997e-05 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832208+0000", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000194928 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.833459+0000", 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0012513240000000001 2024-09-21T15:34:30.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.835987+0000", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002528069 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836013+0000", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5677000000000002e-05 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836047+0000", 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4103999999999997e-05 2024-09-21T15:34:30.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157214 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3887022020000002, 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0049247070000000004, 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157214 2024-09-21T15:34:30.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831637+0000", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831636+0000", 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831642+0000", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.533e-06 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831655+0000", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2724999999999999e-05 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.831662+0000", 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5919999999999997e-06 2024-09-21T15:34:30.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832518+0000", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00085589900000000003 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832605+0000", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7318000000000007e-05 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.832974+0000", 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00036939200000000002 2024-09-21T15:34:30.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.833818+0000", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000843799 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836480+0000", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026621840000000002 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836530+0000", 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9721e-05 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.836562+0000", 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1528e-05 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157272 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3067675210000003, 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006546451, 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157272 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913572+0000", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913569+0000", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913584+0000", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5364000000000001e-05 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913628+0000", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4175999999999999e-05 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913655+0000", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6452999999999999e-05 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913731+0000", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6252000000000001e-05 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913823+0000", 2024-09-21T15:34:30.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2430999999999996e-05 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914239+0000", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00041542199999999998 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918388+0000", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0041490950000000002 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920017+0000", 2024-09-21T15:34:30.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001628839 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920066+0000", 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9453000000000002e-05 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920118+0000", 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1913000000000003e-05 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157273 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3066459449999996, 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0067711760000000003, 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157273 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913693+0000", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913691+0000", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913702+0000", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0305999999999999e-05 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913722+0000", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0013e-05 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913738+0000", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6033e-05 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914805+0000", 2024-09-21T15:34:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001067248 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.915000+0000", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000194744 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.915803+0000", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00080309000000000001 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918394+0000", 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025913780000000001 2024-09-21T15:34:30.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920224+0000", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001830103 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920325+0000", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000101229 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920464+0000", 2024-09-21T15:34:30.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013900199999999999 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157274 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3065873200000002, 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.006850068, 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157274 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913752+0000", 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913751+0000", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913758+0000", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6399999999999997e-06 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913776+0000", 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7677e-05 2024-09-21T15:34:30.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913789+0000", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2643000000000001e-05 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.916645+0000", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0028568629999999999 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.916736+0000", 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0342999999999999e-05 2024-09-21T15:34:30.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917114+0000", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00037831099999999999 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920134+0000", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0030198690000000001 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920524+0000", 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00038972099999999999 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920569+0000", 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5352000000000002e-05 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920602+0000", 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2781999999999999e-05 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.305 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157275 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3065372430000002, 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0070002429999999997, 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157275 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:30.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913802+0000", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913801+0000", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913807+0000", 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.5300000000000002e-06 2024-09-21T15:34:30.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913824+0000", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6444e-05 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.913835+0000", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1636e-05 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917625+0000", 2024-09-21T15:34:30.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0037894550000000002 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917693+0000", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8073999999999999e-05 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.917990+0000", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00029714900000000002 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920616+0000", 2024-09-21T15:34:30.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0026256560000000001 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920670+0000", 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3885999999999999e-05 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920746+0000", 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.648e-05 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920802+0000", 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5940000000000003e-05 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157276 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3057958599999999, 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0073072780000000004, 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.311 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157276 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914543+0000", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914541+0000", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914575+0000", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4075999999999997e-05 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914603+0000", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.747e-05 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914617+0000", 2024-09-21T15:34:30.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4216000000000001e-05 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918460+0000", 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038429660000000002 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.918588+0000", 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128362 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.919111+0000", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052255200000000002 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921667+0000", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0025559570000000002 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921723+0000", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6477999999999997e-05 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921792+0000", 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9283999999999996e-05 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921851+0000", 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.821e-05 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:157277 3.14 3:2d7bac7f:::rbd_object_map.1191d5daa2b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 4.3054344479999997, 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0091388249999999997, 2024-09-21T15:34:30.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 157277 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914905+0000", 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914903+0000", 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914912+0000", 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6449999999999994e-06 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914928+0000", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6413999999999999e-05 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.914939+0000", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0864e-05 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920836+0000", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0058964730000000002 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.920922+0000", 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6340000000000003e-05 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.921264+0000", 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00034240399999999998 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.922131+0000", 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00086667199999999997 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.923949+0000", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0018179649999999999 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.924005+0000", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6431000000000002e-05 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:25.924043+0000", 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8009999999999997e-05 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.321 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.322 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:30.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:30.323 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:30.318+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:30.423 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:30.420+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:30.489 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:30.489 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:30.489 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:30.489 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:30.505 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:30.552 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:30.550+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:30.622 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:30.626 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:30.624+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:30.634 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:30.726 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:30.722+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T15:34:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:34:30.787 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T15:34:30.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.435524103999995, 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:30.809 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.407312927999996, 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:30.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 97.218013560000003, 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.814 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:30.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:30.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:30.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 97.086628868999995, 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:30.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.823 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.925531465000006, 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:30.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:30.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.925478917999996, 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.829 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:30.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:30.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:30.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.925430793000004, 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 93.913033905000006, 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:30.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:30.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.352260923000003, 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.845 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:30.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.850 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.285236557000005, 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:30.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:30.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:30.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:30.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.150116425999997, 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:30.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:30.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:30.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:30.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.109454283000005, 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:30.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:30.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.865 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.109441066000002, 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.109428258999998, 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:30.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.109387851999998, 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.874 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.109363384000005, 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:30.878 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.090141396000007, 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:30.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:30.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:30.886 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.887 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.002837812999999, 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:30.888 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:30.890 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:30.891 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.892 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 24.008152370000001, 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.893 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:30.894 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:30.895 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:30.897 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 23.593893023, 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:30.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:30.901 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:30.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:30.904 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:30.902+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:31.003 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:31.000+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:31.034 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.034 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:31.034 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:31.034 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:31.048 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:31.104 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:31.102+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:31.205 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:31.204+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:31.227 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.227 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:31.228 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:31.228 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:31.228 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:31.228 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:31.229 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:31.230 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-21T15:34:30.753289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":831906004,"num_objects":1301,"num_object_clones":290,"num_object_copies":2602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1301,"num_whiteouts":464,"num_read":185662,"num_read_kb":293215,"num_write":13221,"num_write_kb":288686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":12276010,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10484,"ondisk_log_size":10484,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435256,"kb_used_data":174008,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754539464,"statfs":{"total":773094113280,"available":772648411136,"internally_reserved":0,"allocated":178184192,"data_stored":209228437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18537488,"num_objects":18,"num_object_clones":13,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":2800,"num_read_kb":6582,"num_write":746,"num_write_kb":2914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":5085297,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016534"},"pg_stats":[{"pgid":"3.1f","version":"92'343","reported_seq":4830,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740103+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:26.740103+0000","last_peered":"2024-09-21T15:34:26.740103+0000","last_clean":"2024-09-21T15:34:26.740103+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:26.740103+0000","last_undegraded":"2024-09-21T15:34:26.740103+0000","last_fullsized":"2024-09-21T15:34:26.740103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5635,"num_read_kb":6344,"num_write":497,"num_write_kb":2231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"96'305","reported_seq":4967,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.046833+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:28.046833+0000","last_peered":"2024-09-21T15:34:28.046833+0000","last_clean":"2024-09-21T15:34:28.046833+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:28.046833+0000","last_undegraded":"2024-09-21T15:34:28.046833+0000","last_fullsized":"2024-09-21T15:34:28.046833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":29913554,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":5717,"num_read_kb":13715,"num_write":432,"num_write_kb":25439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"92'126","reported_seq":420,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745245+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:26.745245+0000","last_peered":"2024-09-21T15:34:26.745245+0000","last_clean":"2024-09-21T15:34:26.745245+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:26.745245+0000","last_undegraded":"2024-09-21T15:34:26.745245+0000","last_fullsized":"2024-09-21T15:34:26.745245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":32630107,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":263,"num_read_kb":4854,"num_write":181,"num_write_kb":24546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"93'206","reported_seq":1326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744894+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:26.744894+0000","last_peered":"2024-09-21T15:34:26.744894+0000","last_clean":"2024-09-21T15:34:26.744894+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:26.744894+0000","last_undegraded":"2024-09-21T15:34:26.744894+0000","last_fullsized":"2024-09-21T15:34:26.744894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46862032,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1225,"num_read_kb":4865,"num_write":216,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"92'300","reported_seq":9158,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739708+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:26.739708+0000","last_peered":"2024-09-21T15:34:26.739708+0000","last_clean":"2024-09-21T15:34:26.739708+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:26.739708+0000","last_undegraded":"2024-09-21T15:34:26.739708+0000","last_fullsized":"2024-09-21T15:34:26.739708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":15692760,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":9556,"num_read_kb":9255,"num_write":299,"num_write_kb":3918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"93'173","reported_seq":2267,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.043340+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:28.043340+0000","last_peered":"2024-09-21T15:34:28.043340+0000","last_clean":"2024-09-21T15:34:28.043340+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:28.043340+0000","last_undegraded":"2024-09-21T15:34:28.043340+0000","last_fullsized":"2024-09-21T15:34:28.043340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":17237631,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":2098,"num_read_kb":11372,"num_write":170,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"91'547","reported_seq":33226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744100+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:27.744100+0000","last_peered":"2024-09-21T15:34:27.744100+0000","last_clean":"2024-09-21T15:34:27.744100+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:27.744100+0000","last_undegraded":"2024-09-21T15:34:27.744100+0000","last_fullsized":"2024-09-21T15:34:27.744100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":62210276,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":34584,"num_read_kb":19435,"num_write":784,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"93'389","reported_seq":901,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738718+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:26.738718+0000","last_peered":"2024-09-21T15:34:26.738718+0000","last_clean":"2024-09-21T15:34:26.738718+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:26.738718+0000","last_undegraded":"2024-09-21T15:34:26.738718+0000","last_fullsized":"2024-09-21T15:34:26.738718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":16518862,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":2036,"num_read_kb":4671,"num_write":550,"num_write_kb":13058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"92'575","reported_seq":1771,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740187+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:26.740187+0000","last_peered":"2024-09-21T15:34:26.740187+0000","last_clean":"2024-09-21T15:34:26.740187+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:26.740187+0000","last_undegraded":"2024-09-21T15:34:26.740187+0000","last_fullsized":"2024-09-21T15:34:26.740187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":27191524,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":3844,"num_read_kb":9634,"num_write":881,"num_write_kb":6235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'291","reported_seq":5754,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745730+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:26.745730+0000","last_peered":"2024-09-21T15:34:26.745730+0000","last_clean":"2024-09-21T15:34:26.745730+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:26.745730+0000","last_undegraded":"2024-09-21T15:34:26.745730+0000","last_fullsized":"2024-09-21T15:34:26.745730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'227","reported_seq":4822,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739451+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:34:26.739451+0000","last_peered":"2024-09-21T15:34:26.739451+0000","last_clean":"2024-09-21T15:34:26.739451+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:34:26.739451+0000","last_undegraded":"2024-09-21T15:34:26.739451+0000","last_fullsized":"2024-09-21T15:34:26.739451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":7770401,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4811,"num_read_kb":8549,"num_write":262,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"96'505","reported_seq":1632,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.036249+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:28.036249+0000","last_peered":"2024-09-21T15:34:28.036249+0000","last_clean":"2024-09-21T15:34:28.036249+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:28.036249+0000","last_undegraded":"2024-09-21T15:34:28.036249+0000","last_fullsized":"2024-09-21T15:34:28.036249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":27664823,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":3469,"num_read_kb":5935,"num_write":639,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034812+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:34:27.034812+0000","last_peered":"2024-09-21T15:34:27.034812+0000","last_clean":"2024-09-21T15:34:27.034812+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:34:27.034812+0000","last_undegraded":"2024-09-21T15:34:27.034812+0000","last_fullsized":"2024-09-21T15:34:27.034812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'298","reported_seq":1794,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.971399+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:27.971399+0000","last_peered":"2024-09-21T15:34:27.971399+0000","last_clean":"2024-09-21T15:34:27.971399+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:27.971399+0000","last_undegraded":"2024-09-21T15:34:27.971399+0000","last_fullsized":"2024-09-21T15:34:27.971399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":3634,"num_read_kb":6298,"num_write":279,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.747795+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:27.747795+0000","last_peered":"2024-09-21T15:34:27.747795+0000","last_clean":"2024-09-21T15:34:27.747795+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:27.747795+0000","last_undegraded":"2024-09-21T15:34:27.747795+0000","last_fullsized":"2024-09-21T15:34:27.747795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"93'179","reported_seq":1190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743917+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:26.743917+0000","last_peered":"2024-09-21T15:34:26.743917+0000","last_clean":"2024-09-21T15:34:26.743917+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:26.743917+0000","last_undegraded":"2024-09-21T15:34:26.743917+0000","last_fullsized":"2024-09-21T15:34:26.743917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":35394556,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":1376,"num_read_kb":21000,"num_write":197,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744391+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:26.744391+0000","last_peered":"2024-09-21T15:34:26.744391+0000","last_clean":"2024-09-21T15:34:26.744391+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:26.744391+0000","last_undegraded":"2024-09-21T15:34:26.744391+0000","last_fullsized":"2024-09-21T15:34:26.744391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"93'167","reported_seq":712,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744072+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:27.744072+0000","last_peered":"2024-09-21T15:34:27.744072+0000","last_clean":"2024-09-21T15:34:27.744072+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:27.744072+0000","last_undegraded":"2024-09-21T15:34:27.744072+0000","last_fullsized":"2024-09-21T15:34:27.744072+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":15153076,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":805,"num_read_kb":4007,"num_write":172,"num_write_kb":2694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.745322+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:27.745322+0000","last_peered":"2024-09-21T15:34:27.745322+0000","last_clean":"2024-09-21T15:34:27.745322+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:27.745322+0000","last_undegraded":"2024-09-21T15:34:27.745322+0000","last_fullsized":"2024-09-21T15:34:27.745322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'376","reported_seq":764,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745482+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:26.745482+0000","last_peered":"2024-09-21T15:34:26.745482+0000","last_clean":"2024-09-21T15:34:26.745482+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:26.745482+0000","last_undegraded":"2024-09-21T15:34:26.745482+0000","last_fullsized":"2024-09-21T15:34:26.745482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":12612201,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1449,"num_read_kb":4766,"num_write":486,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"96'180","reported_seq":4694,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.963977+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:27.963977+0000","last_peered":"2024-09-21T15:34:27.963977+0000","last_clean":"2024-09-21T15:34:27.963977+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:27.963977+0000","last_undegraded":"2024-09-21T15:34:27.963977+0000","last_fullsized":"2024-09-21T15:34:27.963977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":22791309,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":4499,"num_read_kb":8806,"num_write":206,"num_write_kb":3971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":93,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744920+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:26.744920+0000","last_peered":"2024-09-21T15:34:26.744920+0000","last_clean":"2024-09-21T15:34:26.744920+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:26.744920+0000","last_undegraded":"2024-09-21T15:34:26.744920+0000","last_fullsized":"2024-09-21T15:34:26.744920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"93'605","reported_seq":1700,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.746714+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:27.746714+0000","last_peered":"2024-09-21T15:34:27.746714+0000","last_clean":"2024-09-21T15:34:27.746714+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:27.746714+0000","last_undegraded":"2024-09-21T15:34:27.746714+0000","last_fullsized":"2024-09-21T15:34:27.746714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25050606,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":3890,"num_read_kb":5348,"num_write":833,"num_write_kb":10490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034215+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:27.034215+0000","last_peered":"2024-09-21T15:34:27.034215+0000","last_clean":"2024-09-21T15:34:27.034215+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:27.034215+0000","last_undegraded":"2024-09-21T15:34:27.034215+0000","last_fullsized":"2024-09-21T15:34:27.034215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"96'390","reported_seq":995,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.071959+0000","last_change":"2024-09-21T15:34:28.072205+0000","last_active":"2024-09-21T15:34:28.071959+0000","last_peered":"2024-09-21T15:34:28.071959+0000","last_clean":"2024-09-21T15:34:26.155429+0000","last_became_active":"2024-09-21T15:34:27.750925+0000","last_became_peered":"2024-09-21T15:34:27.750925+0000","last_unstale":"2024-09-21T15:34:28.071959+0000","last_undegraded":"2024-09-21T15:34:28.012016+0000","last_fullsized":"2024-09-21T15:34:27.739696+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:01.469034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":23935853,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":1785,"num_read_kb":4111,"num_write":605,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745119+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:26.745119+0000","last_peered":"2024-09-21T15:34:26.745119+0000","last_clean":"2024-09-21T15:34:26.745119+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:26.745119+0000","last_undegraded":"2024-09-21T15:34:26.745119+0000","last_fullsized":"2024-09-21T15:34:26.745119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'203","reported_seq":697,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744850+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:26.744850+0000","last_peered":"2024-09-21T15:34:26.744850+0000","last_clean":"2024-09-21T15:34:26.744850+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:26.744850+0000","last_undegraded":"2024-09-21T15:34:26.744850+0000","last_fullsized":"2024-09-21T15:34:26.744850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":23697932,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":985,"num_read_kb":3963,"num_write":190,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":116,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744762+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:26.744762+0000","last_peered":"2024-09-21T15:34:26.744762+0000","last_clean":"2024-09-21T15:34:26.744762+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:26.744762+0000","last_undegraded":"2024-09-21T15:34:26.744762+0000","last_fullsized":"2024-09-21T15:34:26.744762+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738836+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:34:26.738836+0000","last_peered":"2024-09-21T15:34:26.738836+0000","last_clean":"2024-09-21T15:34:26.738836+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:34:26.738836+0000","last_undegraded":"2024-09-21T15:34:26.738836+0000","last_fullsized":"2024-09-21T15:34:26.738836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"96'470","reported_seq":1375,"reported_epoch":97,"state":"peering","last_fresh":"2024-09-21T15:34:28.746290+0000","last_change":"2024-09-21T15:34:28.746824+0000","last_active":"2024-09-21T15:34:28.746290+0000","last_peered":"2024-09-21T15:34:28.016869+0000","last_clean":"2024-09-21T15:34:26.155096+0000","last_became_active":"2024-09-21T15:34:27.758438+0000","last_became_peered":"2024-09-21T15:34:27.758438+0000","last_unstale":"2024-09-21T15:34:28.746290+0000","last_undegraded":"2024-09-21T15:34:28.746290+0000","last_fullsized":"2024-09-21T15:34:28.746290+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:40.070130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":11358204,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":2585,"num_read_kb":6802,"num_write":658,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'447","reported_seq":1488,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739326+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:26.739326+0000","last_peered":"2024-09-21T15:34:26.739326+0000","last_clean":"2024-09-21T15:34:26.739326+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:26.739326+0000","last_undegraded":"2024-09-21T15:34:26.739326+0000","last_fullsized":"2024-09-21T15:34:26.739326+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522171,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":3109,"num_read_kb":7561,"num_write":608,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'260","reported_seq":4744,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034737+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:27.034737+0000","last_peered":"2024-09-21T15:34:27.034737+0000","last_clean":"2024-09-21T15:34:27.034737+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:27.034737+0000","last_undegraded":"2024-09-21T15:34:27.034737+0000","last_fullsized":"2024-09-21T15:34:27.034737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":20964503,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":4747,"num_read_kb":13712,"num_write":284,"num_write_kb":5500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"96'292","reported_seq":1625,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.011167+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:28.011167+0000","last_peered":"2024-09-21T15:34:28.011167+0000","last_clean":"2024-09-21T15:34:28.011167+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:28.011167+0000","last_undegraded":"2024-09-21T15:34:28.011167+0000","last_fullsized":"2024-09-21T15:34:28.011167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":2912,"num_read_kb":10480,"num_write":323,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"96'213","reported_seq":664,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.010912+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:28.010912+0000","last_peered":"2024-09-21T15:34:28.010912+0000","last_clean":"2024-09-21T15:34:28.010912+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:28.010912+0000","last_undegraded":"2024-09-21T15:34:28.010912+0000","last_fullsized":"2024-09-21T15:34:28.010912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":35350897,"num_objects":40,"num_object_clones":14,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":535,"num_read_kb":8299,"num_write":222,"num_write_kb":21178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17290,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744197+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:26.744197+0000","last_peered":"2024-09-21T15:34:26.744197+0000","last_clean":"2024-09-21T15:34:26.744197+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:26.744197+0000","last_undegraded":"2024-09-21T15:34:26.744197+0000","last_fullsized":"2024-09-21T15:34:26.744197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17967,"num_read_kb":4981,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"93'281","reported_seq":33439,"reported_epoch":96,"state":"remapped+peering","last_fresh":"2024-09-21T15:34:27.745921+0000","last_change":"2024-09-21T15:34:27.746048+0000","last_active":"2024-09-21T15:34:26.745033+0000","last_peered":"2024-09-21T15:34:26.191816+0000","last_clean":"2024-09-21T15:34:26.191816+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:34:27.745921+0000","last_undegraded":"2024-09-21T15:34:27.745921+0000","last_fullsized":"2024-09-21T15:34:27.745921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:18.433716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":33827,"num_read_kb":32899,"num_write":316,"num_write_kb":20799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"93'302","reported_seq":1426,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739592+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:34:26.739592+0000","last_peered":"2024-09-21T15:34:26.739592+0000","last_clean":"2024-09-21T15:34:26.739592+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:34:26.739592+0000","last_undegraded":"2024-09-21T15:34:26.739592+0000","last_fullsized":"2024-09-21T15:34:26.739592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":29273859,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":2338,"num_read_kb":6358,"num_write":345,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":727,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743638+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:26.743638+0000","last_peered":"2024-09-21T15:34:26.743638+0000","last_clean":"2024-09-21T15:34:26.743638+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:26.743638+0000","last_undegraded":"2024-09-21T15:34:26.743638+0000","last_fullsized":"2024-09-21T15:34:26.743638+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"96'391","reported_seq":5666,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.072803+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:28.072803+0000","last_peered":"2024-09-21T15:34:28.072803+0000","last_clean":"2024-09-21T15:34:28.072803+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:28.072803+0000","last_undegraded":"2024-09-21T15:34:28.072803+0000","last_fullsized":"2024-09-21T15:34:28.072803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":25206458,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":7269,"num_read_kb":18206,"num_write":524,"num_write_kb":17322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"96'456","reported_seq":5535,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.228586+0000","last_change":"2024-09-21T15:34:28.228848+0000","last_active":"2024-09-21T15:34:28.228586+0000","last_peered":"2024-09-21T15:34:28.228586+0000","last_clean":"2024-09-21T15:34:26.155415+0000","last_became_active":"2024-09-21T15:34:27.760062+0000","last_became_peered":"2024-09-21T15:34:27.760062+0000","last_unstale":"2024-09-21T15:34:28.228586+0000","last_undegraded":"2024-09-21T15:34:27.741167+0000","last_fullsized":"2024-09-21T15:34:27.739664+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:47:48.096630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":19198544,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":12,"num_read":6568,"num_read_kb":6356,"num_write":604,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1099128,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":["0","1","3"],"object_location_counts":[{"shards":"0","objects":42}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"93'311","reported_seq":1811,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743906+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:26.743906+0000","last_peered":"2024-09-21T15:34:26.743906+0000","last_clean":"2024-09-21T15:34:26.743906+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:26.743906+0000","last_undegraded":"2024-09-21T15:34:26.743906+0000","last_fullsized":"2024-09-21T15:34:26.743906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":28644623,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":4098,"num_read_kb":5065,"num_write":300,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":831315617,"num_objects":1297,"num_object_clones":290,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":464,"num_read":185546,"num_read_kb":293115,"num_write":13060,"num_write_kb":286268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":11685642,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167899136,"data_stored":203852805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10364,"ondisk_log_size":10364,"up":64,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":26272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":26902528,"data_stored":30884872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.748}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.88},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":11912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":12197888,"data_stored":13498496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.5},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.625}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.073},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.1120000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0029999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.198},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":24480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":25067520,"data_stored":29579967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.498}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64600000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61576,"kb_used_data":30464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94310264,"statfs":{"total":96636764160,"available":96573710336,"internally_reserved":0,"allocated":31195136,"data_stored":35482439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0389999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.016},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0620000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.80800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75328,"kb_used_data":36012,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94296512,"statfs":{"total":96636764160,"available":96559628288,"internally_reserved":0,"allocated":36876288,"data_stored":45858618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.222}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.316},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.42999999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41996,"kb_used_data":6788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329844,"statfs":{"total":96636764160,"available":96593760256,"internally_reserved":0,"allocated":6950912,"data_stored":7366987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.91100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":15812,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":16191488,"data_stored":18844083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.03},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49296,"kb_used_data":22268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322544,"statfs":{"total":96636764160,"available":96586285056,"internally_reserved":0,"allocated":22802432,"data_stored":27712975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96199999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0189999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.98999999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2649999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.371},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.038}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21069824,"data_stored":26595870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15048704,"data_stored":18317327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5812224,"data_stored":6840250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35737600,"data_stored":45331881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30056448,"data_stored":34955702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23924736,"data_stored":29053211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":12400429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25763840,"data_stored":30358135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:31.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:31.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:31.316 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:31.314+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:31.401 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122522020000002, 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.402 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122497961000001, 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.406 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:31.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:31.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122473998, 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.410 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:31.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.414 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122450231999998, 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:31.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:31.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.418 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.12242659, 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:31.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:31.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:31.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.422 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122401036999999, 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:31.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.426 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122356822, 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:31.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:31.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:31.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122328112999998, 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.434 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.12229657, 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.435 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:31.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122269498000001, 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.439 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.122243565000002, 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.443 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:31.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:31.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.121708622, 2024-09-21T15:34:31.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:31.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.451 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.121078954000001, 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.452 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.117167390999999, 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.456 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:31.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:31.460 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.117120892999999, 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.117071723999999, 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:31.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.117023876000001, 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.469 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:31.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.116975487000001, 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:31.473 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.477 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.116894565999999, 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:31.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:31.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:31.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.113296203000001, 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.482 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.486 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:31.487 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:31.488 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:31.488+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:31.551 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:31.553 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-21T15:34:30.753289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":831906004,"num_objects":1301,"num_object_clones":290,"num_object_copies":2602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1301,"num_whiteouts":464,"num_read":185662,"num_read_kb":293215,"num_write":13221,"num_write_kb":288686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":12276010,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10484,"ondisk_log_size":10484,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435256,"kb_used_data":174008,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754539464,"statfs":{"total":773094113280,"available":772648411136,"internally_reserved":0,"allocated":178184192,"data_stored":209228437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18537488,"num_objects":18,"num_object_clones":13,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":2800,"num_read_kb":6582,"num_write":746,"num_write_kb":2914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":5085297,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016534"},"pg_stats":[{"pgid":"3.1f","version":"92'343","reported_seq":4830,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740103+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:26.740103+0000","last_peered":"2024-09-21T15:34:26.740103+0000","last_clean":"2024-09-21T15:34:26.740103+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:26.740103+0000","last_undegraded":"2024-09-21T15:34:26.740103+0000","last_fullsized":"2024-09-21T15:34:26.740103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5635,"num_read_kb":6344,"num_write":497,"num_write_kb":2231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"96'305","reported_seq":4967,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.046833+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:28.046833+0000","last_peered":"2024-09-21T15:34:28.046833+0000","last_clean":"2024-09-21T15:34:28.046833+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:28.046833+0000","last_undegraded":"2024-09-21T15:34:28.046833+0000","last_fullsized":"2024-09-21T15:34:28.046833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":29913554,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":5717,"num_read_kb":13715,"num_write":432,"num_write_kb":25439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"92'126","reported_seq":420,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745245+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:26.745245+0000","last_peered":"2024-09-21T15:34:26.745245+0000","last_clean":"2024-09-21T15:34:26.745245+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:26.745245+0000","last_undegraded":"2024-09-21T15:34:26.745245+0000","last_fullsized":"2024-09-21T15:34:26.745245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":32630107,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":263,"num_read_kb":4854,"num_write":181,"num_write_kb":24546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"93'206","reported_seq":1326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744894+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:26.744894+0000","last_peered":"2024-09-21T15:34:26.744894+0000","last_clean":"2024-09-21T15:34:26.744894+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:26.744894+0000","last_undegraded":"2024-09-21T15:34:26.744894+0000","last_fullsized":"2024-09-21T15:34:26.744894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46862032,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1225,"num_read_kb":4865,"num_write":216,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"92'300","reported_seq":9158,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739708+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:26.739708+0000","last_peered":"2024-09-21T15:34:26.739708+0000","last_clean":"2024-09-21T15:34:26.739708+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:26.739708+0000","last_undegraded":"2024-09-21T15:34:26.739708+0000","last_fullsized":"2024-09-21T15:34:26.739708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":15692760,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":9556,"num_read_kb":9255,"num_write":299,"num_write_kb":3918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"93'173","reported_seq":2267,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.043340+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:28.043340+0000","last_peered":"2024-09-21T15:34:28.043340+0000","last_clean":"2024-09-21T15:34:28.043340+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:28.043340+0000","last_undegraded":"2024-09-21T15:34:28.043340+0000","last_fullsized":"2024-09-21T15:34:28.043340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":17237631,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":2098,"num_read_kb":11372,"num_write":170,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"91'547","reported_seq":33226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744100+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:27.744100+0000","last_peered":"2024-09-21T15:34:27.744100+0000","last_clean":"2024-09-21T15:34:27.744100+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:27.744100+0000","last_undegraded":"2024-09-21T15:34:27.744100+0000","last_fullsized":"2024-09-21T15:34:27.744100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":62210276,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":34584,"num_read_kb":19435,"num_write":784,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"93'389","reported_seq":901,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738718+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:26.738718+0000","last_peered":"2024-09-21T15:34:26.738718+0000","last_clean":"2024-09-21T15:34:26.738718+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:26.738718+0000","last_undegraded":"2024-09-21T15:34:26.738718+0000","last_fullsized":"2024-09-21T15:34:26.738718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":16518862,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":2036,"num_read_kb":4671,"num_write":550,"num_write_kb":13058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"92'575","reported_seq":1771,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740187+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:26.740187+0000","last_peered":"2024-09-21T15:34:26.740187+0000","last_clean":"2024-09-21T15:34:26.740187+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:26.740187+0000","last_undegraded":"2024-09-21T15:34:26.740187+0000","last_fullsized":"2024-09-21T15:34:26.740187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":27191524,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":3844,"num_read_kb":9634,"num_write":881,"num_write_kb":6235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'291","reported_seq":5754,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745730+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:26.745730+0000","last_peered":"2024-09-21T15:34:26.745730+0000","last_clean":"2024-09-21T15:34:26.745730+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:26.745730+0000","last_undegraded":"2024-09-21T15:34:26.745730+0000","last_fullsized":"2024-09-21T15:34:26.745730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'227","reported_seq":4822,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739451+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:34:26.739451+0000","last_peered":"2024-09-21T15:34:26.739451+0000","last_clean":"2024-09-21T15:34:26.739451+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:34:26.739451+0000","last_undegraded":"2024-09-21T15:34:26.739451+0000","last_fullsized":"2024-09-21T15:34:26.739451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":7770401,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4811,"num_read_kb":8549,"num_write":262,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"96'505","reported_seq":1632,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.036249+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:28.036249+0000","last_peered":"2024-09-21T15:34:28.036249+0000","last_clean":"2024-09-21T15:34:28.036249+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:28.036249+0000","last_undegraded":"2024-09-21T15:34:28.036249+0000","last_fullsized":"2024-09-21T15:34:28.036249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":27664823,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":3469,"num_read_kb":5935,"num_write":639,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034812+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:34:27.034812+0000","last_peered":"2024-09-21T15:34:27.034812+0000","last_clean":"2024-09-21T15:34:27.034812+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:34:27.034812+0000","last_undegraded":"2024-09-21T15:34:27.034812+0000","last_fullsized":"2024-09-21T15:34:27.034812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'298","reported_seq":1794,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.971399+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:27.971399+0000","last_peered":"2024-09-21T15:34:27.971399+0000","last_clean":"2024-09-21T15:34:27.971399+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:27.971399+0000","last_undegraded":"2024-09-21T15:34:27.971399+0000","last_fullsized":"2024-09-21T15:34:27.971399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":3634,"num_read_kb":6298,"num_write":279,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.747795+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:27.747795+0000","last_peered":"2024-09-21T15:34:27.747795+0000","last_clean":"2024-09-21T15:34:27.747795+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:27.747795+0000","last_undegraded":"2024-09-21T15:34:27.747795+0000","last_fullsized":"2024-09-21T15:34:27.747795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"93'179","reported_seq":1190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743917+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:26.743917+0000","last_peered":"2024-09-21T15:34:26.743917+0000","last_clean":"2024-09-21T15:34:26.743917+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:26.743917+0000","last_undegraded":"2024-09-21T15:34:26.743917+0000","last_fullsized":"2024-09-21T15:34:26.743917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":35394556,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":1376,"num_read_kb":21000,"num_write":197,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744391+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:26.744391+0000","last_peered":"2024-09-21T15:34:26.744391+0000","last_clean":"2024-09-21T15:34:26.744391+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:26.744391+0000","last_undegraded":"2024-09-21T15:34:26.744391+0000","last_fullsized":"2024-09-21T15:34:26.744391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"93'167","reported_seq":712,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744072+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:27.744072+0000","last_peered":"2024-09-21T15:34:27.744072+0000","last_clean":"2024-09-21T15:34:27.744072+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:27.744072+0000","last_undegraded":"2024-09-21T15:34:27.744072+0000","last_fullsized":"2024-09-21T15:34:27.744072+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":15153076,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":805,"num_read_kb":4007,"num_write":172,"num_write_kb":2694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.745322+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:27.745322+0000","last_peered":"2024-09-21T15:34:27.745322+0000","last_clean":"2024-09-21T15:34:27.745322+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:27.745322+0000","last_undegraded":"2024-09-21T15:34:27.745322+0000","last_fullsized":"2024-09-21T15:34:27.745322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'376","reported_seq":764,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745482+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:26.745482+0000","last_peered":"2024-09-21T15:34:26.745482+0000","last_clean":"2024-09-21T15:34:26.745482+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:26.745482+0000","last_undegraded":"2024-09-21T15:34:26.745482+0000","last_fullsized":"2024-09-21T15:34:26.745482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":12612201,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1449,"num_read_kb":4766,"num_write":486,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"96'180","reported_seq":4694,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.963977+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:27.963977+0000","last_peered":"2024-09-21T15:34:27.963977+0000","last_clean":"2024-09-21T15:34:27.963977+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:27.963977+0000","last_undegraded":"2024-09-21T15:34:27.963977+0000","last_fullsized":"2024-09-21T15:34:27.963977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":22791309,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":4499,"num_read_kb":8806,"num_write":206,"num_write_kb":3971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":93,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744920+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:26.744920+0000","last_peered":"2024-09-21T15:34:26.744920+0000","last_clean":"2024-09-21T15:34:26.744920+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:26.744920+0000","last_undegraded":"2024-09-21T15:34:26.744920+0000","last_fullsized":"2024-09-21T15:34:26.744920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"93'605","reported_seq":1700,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.746714+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:27.746714+0000","last_peered":"2024-09-21T15:34:27.746714+0000","last_clean":"2024-09-21T15:34:27.746714+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:27.746714+0000","last_undegraded":"2024-09-21T15:34:27.746714+0000","last_fullsized":"2024-09-21T15:34:27.746714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25050606,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":3890,"num_read_kb":5348,"num_write":833,"num_write_kb":10490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034215+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:27.034215+0000","last_peered":"2024-09-21T15:34:27.034215+0000","last_clean":"2024-09-21T15:34:27.034215+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:27.034215+0000","last_undegraded":"2024-09-21T15:34:27.034215+0000","last_fullsized":"2024-09-21T15:34:27.034215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"96'390","reported_seq":995,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.071959+0000","last_change":"2024-09-21T15:34:28.072205+0000","last_active":"2024-09-21T15:34:28.071959+0000","last_peered":"2024-09-21T15:34:28.071959+0000","last_clean":"2024-09-21T15:34:26.155429+0000","last_became_active":"2024-09-21T15:34:27.750925+0000","last_became_peered":"2024-09-21T15:34:27.750925+0000","last_unstale":"2024-09-21T15:34:28.071959+0000","last_undegraded":"2024-09-21T15:34:28.012016+0000","last_fullsized":"2024-09-21T15:34:27.739696+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:01.469034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":23935853,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":1785,"num_read_kb":4111,"num_write":605,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745119+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:26.745119+0000","last_peered":"2024-09-21T15:34:26.745119+0000","last_clean":"2024-09-21T15:34:26.745119+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:26.745119+0000","last_undegraded":"2024-09-21T15:34:26.745119+0000","last_fullsized":"2024-09-21T15:34:26.745119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'203","reported_seq":697,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744850+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:26.744850+0000","last_peered":"2024-09-21T15:34:26.744850+0000","last_clean":"2024-09-21T15:34:26.744850+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:26.744850+0000","last_undegraded":"2024-09-21T15:34:26.744850+0000","last_fullsized":"2024-09-21T15:34:26.744850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":23697932,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":985,"num_read_kb":3963,"num_write":190,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":116,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744762+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:26.744762+0000","last_peered":"2024-09-21T15:34:26.744762+0000","last_clean":"2024-09-21T15:34:26.744762+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:26.744762+0000","last_undegraded":"2024-09-21T15:34:26.744762+0000","last_fullsized":"2024-09-21T15:34:26.744762+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738836+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:34:26.738836+0000","last_peered":"2024-09-21T15:34:26.738836+0000","last_clean":"2024-09-21T15:34:26.738836+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:34:26.738836+0000","last_undegraded":"2024-09-21T15:34:26.738836+0000","last_fullsized":"2024-09-21T15:34:26.738836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"96'470","reported_seq":1375,"reported_epoch":97,"state":"peering","last_fresh":"2024-09-21T15:34:28.746290+0000","last_change":"2024-09-21T15:34:28.746824+0000","last_active":"2024-09-21T15:34:28.746290+0000","last_peered":"2024-09-21T15:34:28.016869+0000","last_clean":"2024-09-21T15:34:26.155096+0000","last_became_active":"2024-09-21T15:34:27.758438+0000","last_became_peered":"2024-09-21T15:34:27.758438+0000","last_unstale":"2024-09-21T15:34:28.746290+0000","last_undegraded":"2024-09-21T15:34:28.746290+0000","last_fullsized":"2024-09-21T15:34:28.746290+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:40.070130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":11358204,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":2585,"num_read_kb":6802,"num_write":658,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'447","reported_seq":1488,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739326+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:26.739326+0000","last_peered":"2024-09-21T15:34:26.739326+0000","last_clean":"2024-09-21T15:34:26.739326+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:26.739326+0000","last_undegraded":"2024-09-21T15:34:26.739326+0000","last_fullsized":"2024-09-21T15:34:26.739326+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522171,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":3109,"num_read_kb":7561,"num_write":608,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'260","reported_seq":4744,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034737+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:27.034737+0000","last_peered":"2024-09-21T15:34:27.034737+0000","last_clean":"2024-09-21T15:34:27.034737+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:27.034737+0000","last_undegraded":"2024-09-21T15:34:27.034737+0000","last_fullsized":"2024-09-21T15:34:27.034737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":20964503,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":4747,"num_read_kb":13712,"num_write":284,"num_write_kb":5500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"96'292","reported_seq":1625,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.011167+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:28.011167+0000","last_peered":"2024-09-21T15:34:28.011167+0000","last_clean":"2024-09-21T15:34:28.011167+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:28.011167+0000","last_undegraded":"2024-09-21T15:34:28.011167+0000","last_fullsized":"2024-09-21T15:34:28.011167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":2912,"num_read_kb":10480,"num_write":323,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"96'213","reported_seq":664,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.010912+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:28.010912+0000","last_peered":"2024-09-21T15:34:28.010912+0000","last_clean":"2024-09-21T15:34:28.010912+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:28.010912+0000","last_undegraded":"2024-09-21T15:34:28.010912+0000","last_fullsized":"2024-09-21T15:34:28.010912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":35350897,"num_objects":40,"num_object_clones":14,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":535,"num_read_kb":8299,"num_write":222,"num_write_kb":21178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17290,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744197+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:26.744197+0000","last_peered":"2024-09-21T15:34:26.744197+0000","last_clean":"2024-09-21T15:34:26.744197+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:26.744197+0000","last_undegraded":"2024-09-21T15:34:26.744197+0000","last_fullsized":"2024-09-21T15:34:26.744197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17967,"num_read_kb":4981,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"93'281","reported_seq":33439,"reported_epoch":96,"state":"remapped+peering","last_fresh":"2024-09-21T15:34:27.745921+0000","last_change":"2024-09-21T15:34:27.746048+0000","last_active":"2024-09-21T15:34:26.745033+0000","last_peered":"2024-09-21T15:34:26.191816+0000","last_clean":"2024-09-21T15:34:26.191816+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:34:27.745921+0000","last_undegraded":"2024-09-21T15:34:27.745921+0000","last_fullsized":"2024-09-21T15:34:27.745921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:18.433716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":33827,"num_read_kb":32899,"num_write":316,"num_write_kb":20799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"93'302","reported_seq":1426,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739592+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:34:26.739592+0000","last_peered":"2024-09-21T15:34:26.739592+0000","last_clean":"2024-09-21T15:34:26.739592+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:34:26.739592+0000","last_undegraded":"2024-09-21T15:34:26.739592+0000","last_fullsized":"2024-09-21T15:34:26.739592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":29273859,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":2338,"num_read_kb":6358,"num_write":345,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":727,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743638+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:26.743638+0000","last_peered":"2024-09-21T15:34:26.743638+0000","last_clean":"2024-09-21T15:34:26.743638+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:26.743638+0000","last_undegraded":"2024-09-21T15:34:26.743638+0000","last_fullsized":"2024-09-21T15:34:26.743638+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"96'391","reported_seq":5666,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.072803+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:28.072803+0000","last_peered":"2024-09-21T15:34:28.072803+0000","last_clean":"2024-09-21T15:34:28.072803+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:28.072803+0000","last_undegraded":"2024-09-21T15:34:28.072803+0000","last_fullsized":"2024-09-21T15:34:28.072803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":25206458,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":7269,"num_read_kb":18206,"num_write":524,"num_write_kb":17322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"96'456","reported_seq":5535,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.228586+0000","last_change":"2024-09-21T15:34:28.228848+0000","last_active":"2024-09-21T15:34:28.228586+0000","last_peered":"2024-09-21T15:34:28.228586+0000","last_clean":"2024-09-21T15:34:26.155415+0000","last_became_active":"2024-09-21T15:34:27.760062+0000","last_became_peered":"2024-09-21T15:34:27.760062+0000","last_unstale":"2024-09-21T15:34:28.228586+0000","last_undegraded":"2024-09-21T15:34:27.741167+0000","last_fullsized":"2024-09-21T15:34:27.739664+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:47:48.096630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":19198544,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":12,"num_read":6568,"num_read_kb":6356,"num_write":604,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1099128,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":["0","1","3"],"object_location_counts":[{"shards":"0","objects":42}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"93'311","reported_seq":1811,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743906+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:26.743906+0000","last_peered":"2024-09-21T15:34:26.743906+0000","last_clean":"2024-09-21T15:34:26.743906+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:26.743906+0000","last_undegraded":"2024-09-21T15:34:26.743906+0000","last_fullsized":"2024-09-21T15:34:26.743906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":28644623,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":4098,"num_read_kb":5065,"num_write":300,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":831315617,"num_objects":1297,"num_object_clones":290,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":464,"num_read":185546,"num_read_kb":293115,"num_write":13060,"num_write_kb":286268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":11685642,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167899136,"data_stored":203852805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10364,"ondisk_log_size":10364,"up":64,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":26272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":26902528,"data_stored":30884872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.748}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.88},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":11912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":12197888,"data_stored":13498496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.5},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.625}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.073},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.1120000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0029999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.198},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":24480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":25067520,"data_stored":29579967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.498}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64600000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61576,"kb_used_data":30464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94310264,"statfs":{"total":96636764160,"available":96573710336,"internally_reserved":0,"allocated":31195136,"data_stored":35482439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0389999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.016},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0620000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.80800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75328,"kb_used_data":36012,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94296512,"statfs":{"total":96636764160,"available":96559628288,"internally_reserved":0,"allocated":36876288,"data_stored":45858618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.222}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.316},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.42999999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41996,"kb_used_data":6788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329844,"statfs":{"total":96636764160,"available":96593760256,"internally_reserved":0,"allocated":6950912,"data_stored":7366987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.91100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":15812,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":16191488,"data_stored":18844083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.03},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49296,"kb_used_data":22268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322544,"statfs":{"total":96636764160,"available":96586285056,"internally_reserved":0,"allocated":22802432,"data_stored":27712975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96199999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0189999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.98999999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2649999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.371},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.038}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21069824,"data_stored":26595870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15048704,"data_stored":18317327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5812224,"data_stored":6840250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35737600,"data_stored":45331881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30056448,"data_stored":34955702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23924736,"data_stored":29053211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":12400429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25763840,"data_stored":30358135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:31.572 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T15:34:31.612 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:31.610+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:31.649 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.649 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:31.650 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:31.650 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:31.664 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:31.689 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:31.686+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:31.790 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:31.788+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:31.791 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.791 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:31.791 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:31.791 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:31.791 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:31.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:31.893 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:31.892+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:31.967 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T15:34:31.974 INFO:teuthology.orchestra.run.smithi107.stdout:4910 read 0xd2cc390 thru 0xd2d70ee (0xad5f bytes) 2024-09-21T15:34:31.975 INFO:teuthology.orchestra.run.smithi107.stdout:4912 writesame 0x67ceead thru 0x67d951c data_size 0x8(0xa670 bytes) 2024-09-21T15:34:31.981 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.7003658, 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:34:31.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.986 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.700307239000001, 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:31.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.700236524000001, 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:34:31.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.700186889000001, 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:31.995 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:31.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:31.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:31.999 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.700119121, 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:34:32.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.700008248, 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.007 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699914281000002, 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.008 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:34:32.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.011 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699866155999999, 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.012 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:34:32.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699796478, 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.016 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:34:32.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:34:32.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699747085999999, 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:34:32.020 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.024 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699700217, 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699653122000001, 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699604084000001, 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699536336000001, 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.040 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699487656999999, 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.044 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.699377374000001, 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.697128917000001, 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:34:32.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:34:32.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.697059165999999, 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:32.057 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.696969514999999, 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.061 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 24.696689732999999, 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:32.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:32.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:32.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:32.066 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:32.066 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:32.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:32.067 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:32.067 INFO:teuthology.orchestra.run.smithi107.stdout:{"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":135,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1726932748,"num_in_osds":8,"osd_in_since":1726932800,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"active+undersized+remapped","count":2},{"state_name":"remapped+peering","count":1},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1301,"data_bytes":831906004,"bytes_used":445702144,"bytes_avail":772648411136,"bytes_total":773094113280,"inactive_pgs_ratio":0.048780485987663269,"recovering_objects_per_sec":48,"recovering_bytes_per_sec":1270974,"recovering_keys_per_sec":5,"num_objects_recovered":193,"num_bytes_recovered":5085297,"num_keys_recovered":23,"read_bytes_sec":1684528,"write_bytes_sec":745778,"read_op_per_sec":699,"write_op_per_sec":186},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-21T15:33:29.652795+0000","services":{}},"progress_events":{}} 2024-09-21T15:34:32.068 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:32.068 INFO:teuthology.orchestra.run.smithi107.stdout:4914 write 0x2d904a7 thru 0x2d9b701 (0xb25b bytes) 2024-09-21T15:34:32.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:32.069 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:32.064+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:32.069 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T15:34:32.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:32.070 INFO:teuthology.orchestra.run.smithi107.stdout:4915 write 0xb60d1ba thru 0xb61a752 (0xd599 bytes) 2024-09-21T15:34:32.070 INFO:teuthology.orchestra.run.smithi107.stdout:4917 flatten 2024-09-21T15:34:32.168 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:32.166+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:32.269 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:32.266+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:32.272 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:32.272 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:32.272 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:32.272 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:32.295 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:32.313 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:32.343 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:32.370 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:32.368+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:32.375 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:32.375 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:32.404 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:32.405 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:32.425 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:32.425 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:32.425 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:32.425 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:32.425 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:32.432 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:32.433 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:32.442 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:32.462 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:32.463 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:32.471 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:32.466+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:32.487 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:32.487 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:32.515 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:32.515 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:32.541 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:32.542 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:32.565 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-21T15:34:31.773921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":831906004,"num_objects":1301,"num_object_clones":290,"num_object_copies":2602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1301,"num_whiteouts":464,"num_read":185662,"num_read_kb":293215,"num_write":13221,"num_write_kb":288686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":12276010,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10484,"ondisk_log_size":10484,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435256,"kb_used_data":174008,"kb_used_omap":12,"kb_used_meta":261107,"kb_avail":754539464,"statfs":{"total":773094113280,"available":772648411136,"internally_reserved":0,"allocated":178184192,"data_stored":209228437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":267374137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18537488,"num_objects":18,"num_object_clones":13,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":2800,"num_read_kb":6582,"num_write":746,"num_write_kb":2914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":5085297,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035405"},"pg_stats":[{"pgid":"3.1f","version":"92'343","reported_seq":4830,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740103+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:26.740103+0000","last_peered":"2024-09-21T15:34:26.740103+0000","last_clean":"2024-09-21T15:34:26.740103+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:26.740103+0000","last_undegraded":"2024-09-21T15:34:26.740103+0000","last_fullsized":"2024-09-21T15:34:26.740103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5635,"num_read_kb":6344,"num_write":497,"num_write_kb":2231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"96'305","reported_seq":4967,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.046833+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:28.046833+0000","last_peered":"2024-09-21T15:34:28.046833+0000","last_clean":"2024-09-21T15:34:28.046833+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:28.046833+0000","last_undegraded":"2024-09-21T15:34:28.046833+0000","last_fullsized":"2024-09-21T15:34:28.046833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":29913554,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":5717,"num_read_kb":13715,"num_write":432,"num_write_kb":25439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"92'126","reported_seq":420,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745245+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:26.745245+0000","last_peered":"2024-09-21T15:34:26.745245+0000","last_clean":"2024-09-21T15:34:26.745245+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:26.745245+0000","last_undegraded":"2024-09-21T15:34:26.745245+0000","last_fullsized":"2024-09-21T15:34:26.745245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":32630107,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":263,"num_read_kb":4854,"num_write":181,"num_write_kb":24546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"93'206","reported_seq":1326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744894+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:26.744894+0000","last_peered":"2024-09-21T15:34:26.744894+0000","last_clean":"2024-09-21T15:34:26.744894+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:26.744894+0000","last_undegraded":"2024-09-21T15:34:26.744894+0000","last_fullsized":"2024-09-21T15:34:26.744894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46862032,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1225,"num_read_kb":4865,"num_write":216,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"92'300","reported_seq":9158,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739708+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:26.739708+0000","last_peered":"2024-09-21T15:34:26.739708+0000","last_clean":"2024-09-21T15:34:26.739708+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:26.739708+0000","last_undegraded":"2024-09-21T15:34:26.739708+0000","last_fullsized":"2024-09-21T15:34:26.739708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":15692760,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":9556,"num_read_kb":9255,"num_write":299,"num_write_kb":3918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"93'173","reported_seq":2267,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.043340+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:28.043340+0000","last_peered":"2024-09-21T15:34:28.043340+0000","last_clean":"2024-09-21T15:34:28.043340+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:28.043340+0000","last_undegraded":"2024-09-21T15:34:28.043340+0000","last_fullsized":"2024-09-21T15:34:28.043340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":17237631,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":2098,"num_read_kb":11372,"num_write":170,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"91'547","reported_seq":33226,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744100+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:27.744100+0000","last_peered":"2024-09-21T15:34:27.744100+0000","last_clean":"2024-09-21T15:34:27.744100+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:27.744100+0000","last_undegraded":"2024-09-21T15:34:27.744100+0000","last_fullsized":"2024-09-21T15:34:27.744100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":62210276,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":34584,"num_read_kb":19435,"num_write":784,"num_write_kb":4925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"93'389","reported_seq":901,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738718+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:26.738718+0000","last_peered":"2024-09-21T15:34:26.738718+0000","last_clean":"2024-09-21T15:34:26.738718+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:26.738718+0000","last_undegraded":"2024-09-21T15:34:26.738718+0000","last_fullsized":"2024-09-21T15:34:26.738718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":16518862,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":2036,"num_read_kb":4671,"num_write":550,"num_write_kb":13058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"92'575","reported_seq":1771,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.740187+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:26.740187+0000","last_peered":"2024-09-21T15:34:26.740187+0000","last_clean":"2024-09-21T15:34:26.740187+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:26.740187+0000","last_undegraded":"2024-09-21T15:34:26.740187+0000","last_fullsized":"2024-09-21T15:34:26.740187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":27191524,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":3844,"num_read_kb":9634,"num_write":881,"num_write_kb":6235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'291","reported_seq":5754,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745730+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:26.745730+0000","last_peered":"2024-09-21T15:34:26.745730+0000","last_clean":"2024-09-21T15:34:26.745730+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:26.745730+0000","last_undegraded":"2024-09-21T15:34:26.745730+0000","last_fullsized":"2024-09-21T15:34:26.745730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'227","reported_seq":4822,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739451+0000","last_change":"2024-09-21T15:33:31.739919+0000","last_active":"2024-09-21T15:34:26.739451+0000","last_peered":"2024-09-21T15:34:26.739451+0000","last_clean":"2024-09-21T15:34:26.739451+0000","last_became_active":"2024-09-21T15:33:31.738150+0000","last_became_peered":"2024-09-21T15:33:31.738150+0000","last_unstale":"2024-09-21T15:34:26.739451+0000","last_undegraded":"2024-09-21T15:34:26.739451+0000","last_fullsized":"2024-09-21T15:34:26.739451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:11:31.407626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":7770401,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":4811,"num_read_kb":8549,"num_write":262,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"96'505","reported_seq":1632,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.036249+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:28.036249+0000","last_peered":"2024-09-21T15:34:28.036249+0000","last_clean":"2024-09-21T15:34:28.036249+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:28.036249+0000","last_undegraded":"2024-09-21T15:34:28.036249+0000","last_fullsized":"2024-09-21T15:34:28.036249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":27664823,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":3469,"num_read_kb":5935,"num_write":639,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":99,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034812+0000","last_change":"2024-09-21T15:33:26.798259+0000","last_active":"2024-09-21T15:34:27.034812+0000","last_peered":"2024-09-21T15:34:27.034812+0000","last_clean":"2024-09-21T15:34:27.034812+0000","last_became_active":"2024-09-21T15:33:26.797886+0000","last_became_peered":"2024-09-21T15:33:26.797886+0000","last_unstale":"2024-09-21T15:34:27.034812+0000","last_undegraded":"2024-09-21T15:34:27.034812+0000","last_fullsized":"2024-09-21T15:34:27.034812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:21:29.228348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'298","reported_seq":1794,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.971399+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:27.971399+0000","last_peered":"2024-09-21T15:34:27.971399+0000","last_clean":"2024-09-21T15:34:27.971399+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:27.971399+0000","last_undegraded":"2024-09-21T15:34:27.971399+0000","last_fullsized":"2024-09-21T15:34:27.971399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":3634,"num_read_kb":6298,"num_write":279,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.747795+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:27.747795+0000","last_peered":"2024-09-21T15:34:27.747795+0000","last_clean":"2024-09-21T15:34:27.747795+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:27.747795+0000","last_undegraded":"2024-09-21T15:34:27.747795+0000","last_fullsized":"2024-09-21T15:34:27.747795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"93'179","reported_seq":1190,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743917+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:26.743917+0000","last_peered":"2024-09-21T15:34:26.743917+0000","last_clean":"2024-09-21T15:34:26.743917+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:26.743917+0000","last_undegraded":"2024-09-21T15:34:26.743917+0000","last_fullsized":"2024-09-21T15:34:26.743917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":35394556,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":1376,"num_read_kb":21000,"num_write":197,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744391+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:26.744391+0000","last_peered":"2024-09-21T15:34:26.744391+0000","last_clean":"2024-09-21T15:34:26.744391+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:26.744391+0000","last_undegraded":"2024-09-21T15:34:26.744391+0000","last_fullsized":"2024-09-21T15:34:26.744391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"93'167","reported_seq":712,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.744072+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:27.744072+0000","last_peered":"2024-09-21T15:34:27.744072+0000","last_clean":"2024-09-21T15:34:27.744072+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:27.744072+0000","last_undegraded":"2024-09-21T15:34:27.744072+0000","last_fullsized":"2024-09-21T15:34:27.744072+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":15153076,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":805,"num_read_kb":4007,"num_write":172,"num_write_kb":2694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.745322+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:27.745322+0000","last_peered":"2024-09-21T15:34:27.745322+0000","last_clean":"2024-09-21T15:34:27.745322+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:27.745322+0000","last_undegraded":"2024-09-21T15:34:27.745322+0000","last_fullsized":"2024-09-21T15:34:27.745322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'376","reported_seq":764,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745482+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:26.745482+0000","last_peered":"2024-09-21T15:34:26.745482+0000","last_clean":"2024-09-21T15:34:26.745482+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:26.745482+0000","last_undegraded":"2024-09-21T15:34:26.745482+0000","last_fullsized":"2024-09-21T15:34:26.745482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":12612201,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1449,"num_read_kb":4766,"num_write":486,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"96'180","reported_seq":4694,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.963977+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:27.963977+0000","last_peered":"2024-09-21T15:34:27.963977+0000","last_clean":"2024-09-21T15:34:27.963977+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:27.963977+0000","last_undegraded":"2024-09-21T15:34:27.963977+0000","last_fullsized":"2024-09-21T15:34:27.963977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":22791309,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":4499,"num_read_kb":8806,"num_write":206,"num_write_kb":3971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":93,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744920+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:26.744920+0000","last_peered":"2024-09-21T15:34:26.744920+0000","last_clean":"2024-09-21T15:34:26.744920+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:26.744920+0000","last_undegraded":"2024-09-21T15:34:26.744920+0000","last_fullsized":"2024-09-21T15:34:26.744920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"93'605","reported_seq":1700,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.746714+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:27.746714+0000","last_peered":"2024-09-21T15:34:27.746714+0000","last_clean":"2024-09-21T15:34:27.746714+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:27.746714+0000","last_undegraded":"2024-09-21T15:34:27.746714+0000","last_fullsized":"2024-09-21T15:34:27.746714+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25050606,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":3890,"num_read_kb":5348,"num_write":833,"num_write_kb":10490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034215+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:27.034215+0000","last_peered":"2024-09-21T15:34:27.034215+0000","last_clean":"2024-09-21T15:34:27.034215+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:27.034215+0000","last_undegraded":"2024-09-21T15:34:27.034215+0000","last_fullsized":"2024-09-21T15:34:27.034215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"96'390","reported_seq":995,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.071959+0000","last_change":"2024-09-21T15:34:28.072205+0000","last_active":"2024-09-21T15:34:28.071959+0000","last_peered":"2024-09-21T15:34:28.071959+0000","last_clean":"2024-09-21T15:34:26.155429+0000","last_became_active":"2024-09-21T15:34:27.750925+0000","last_became_peered":"2024-09-21T15:34:27.750925+0000","last_unstale":"2024-09-21T15:34:28.071959+0000","last_undegraded":"2024-09-21T15:34:28.012016+0000","last_fullsized":"2024-09-21T15:34:27.739696+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:01.469034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":23935853,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":1785,"num_read_kb":4111,"num_write":605,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":92,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.745119+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:26.745119+0000","last_peered":"2024-09-21T15:34:26.745119+0000","last_clean":"2024-09-21T15:34:26.745119+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:26.745119+0000","last_undegraded":"2024-09-21T15:34:26.745119+0000","last_fullsized":"2024-09-21T15:34:26.745119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'203","reported_seq":697,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744850+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:26.744850+0000","last_peered":"2024-09-21T15:34:26.744850+0000","last_clean":"2024-09-21T15:34:26.744850+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:26.744850+0000","last_undegraded":"2024-09-21T15:34:26.744850+0000","last_fullsized":"2024-09-21T15:34:26.744850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":23697932,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":985,"num_read_kb":3963,"num_write":190,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":116,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744762+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:26.744762+0000","last_peered":"2024-09-21T15:34:26.744762+0000","last_clean":"2024-09-21T15:34:26.744762+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:26.744762+0000","last_undegraded":"2024-09-21T15:34:26.744762+0000","last_fullsized":"2024-09-21T15:34:26.744762+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.738836+0000","last_change":"2024-09-21T15:33:32.736734+0000","last_active":"2024-09-21T15:34:26.738836+0000","last_peered":"2024-09-21T15:34:26.738836+0000","last_clean":"2024-09-21T15:34:26.738836+0000","last_became_active":"2024-09-21T15:33:32.736295+0000","last_became_peered":"2024-09-21T15:33:32.736295+0000","last_unstale":"2024-09-21T15:34:26.738836+0000","last_undegraded":"2024-09-21T15:34:26.738836+0000","last_fullsized":"2024-09-21T15:34:26.738836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:32.977973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"96'470","reported_seq":1375,"reported_epoch":97,"state":"peering","last_fresh":"2024-09-21T15:34:28.746290+0000","last_change":"2024-09-21T15:34:28.746824+0000","last_active":"2024-09-21T15:34:28.746290+0000","last_peered":"2024-09-21T15:34:28.016869+0000","last_clean":"2024-09-21T15:34:26.155096+0000","last_became_active":"2024-09-21T15:34:27.758438+0000","last_became_peered":"2024-09-21T15:34:27.758438+0000","last_unstale":"2024-09-21T15:34:28.746290+0000","last_undegraded":"2024-09-21T15:34:28.746290+0000","last_fullsized":"2024-09-21T15:34:28.746290+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:40.070130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":11358204,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":2585,"num_read_kb":6802,"num_write":658,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'447","reported_seq":1488,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739326+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:26.739326+0000","last_peered":"2024-09-21T15:34:26.739326+0000","last_clean":"2024-09-21T15:34:26.739326+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:26.739326+0000","last_undegraded":"2024-09-21T15:34:26.739326+0000","last_fullsized":"2024-09-21T15:34:26.739326+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522171,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":3109,"num_read_kb":7561,"num_write":608,"num_write_kb":3663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"93'260","reported_seq":4744,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:27.034737+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:27.034737+0000","last_peered":"2024-09-21T15:34:27.034737+0000","last_clean":"2024-09-21T15:34:27.034737+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:27.034737+0000","last_undegraded":"2024-09-21T15:34:27.034737+0000","last_fullsized":"2024-09-21T15:34:27.034737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":20964503,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":4747,"num_read_kb":13712,"num_write":284,"num_write_kb":5500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"96'292","reported_seq":1625,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.011167+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:28.011167+0000","last_peered":"2024-09-21T15:34:28.011167+0000","last_clean":"2024-09-21T15:34:28.011167+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:28.011167+0000","last_undegraded":"2024-09-21T15:34:28.011167+0000","last_fullsized":"2024-09-21T15:34:28.011167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":2912,"num_read_kb":10480,"num_write":323,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"96'213","reported_seq":664,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.010912+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:28.010912+0000","last_peered":"2024-09-21T15:34:28.010912+0000","last_clean":"2024-09-21T15:34:28.010912+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:28.010912+0000","last_undegraded":"2024-09-21T15:34:28.010912+0000","last_fullsized":"2024-09-21T15:34:28.010912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":35350897,"num_objects":40,"num_object_clones":14,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":535,"num_read_kb":8299,"num_write":222,"num_write_kb":21178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17290,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.744197+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:26.744197+0000","last_peered":"2024-09-21T15:34:26.744197+0000","last_clean":"2024-09-21T15:34:26.744197+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:26.744197+0000","last_undegraded":"2024-09-21T15:34:26.744197+0000","last_fullsized":"2024-09-21T15:34:26.744197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17967,"num_read_kb":4981,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"93'281","reported_seq":33439,"reported_epoch":96,"state":"remapped+peering","last_fresh":"2024-09-21T15:34:27.745921+0000","last_change":"2024-09-21T15:34:27.746048+0000","last_active":"2024-09-21T15:34:26.745033+0000","last_peered":"2024-09-21T15:34:26.191816+0000","last_clean":"2024-09-21T15:34:26.191816+0000","last_became_active":"2024-09-21T15:33:29.755002+0000","last_became_peered":"2024-09-21T15:33:29.755002+0000","last_unstale":"2024-09-21T15:34:27.745921+0000","last_undegraded":"2024-09-21T15:34:27.745921+0000","last_fullsized":"2024-09-21T15:34:27.745921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:18.433716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":33827,"num_read_kb":32899,"num_write":316,"num_write_kb":20799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1175137,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"93'302","reported_seq":1426,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.739592+0000","last_change":"2024-09-21T15:33:32.740961+0000","last_active":"2024-09-21T15:34:26.739592+0000","last_peered":"2024-09-21T15:34:26.739592+0000","last_clean":"2024-09-21T15:34:26.739592+0000","last_became_active":"2024-09-21T15:33:32.737904+0000","last_became_peered":"2024-09-21T15:33:32.737904+0000","last_unstale":"2024-09-21T15:34:26.739592+0000","last_undegraded":"2024-09-21T15:34:26.739592+0000","last_fullsized":"2024-09-21T15:34:26.739592+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:28.115531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":29273859,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":2338,"num_read_kb":6358,"num_write":345,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":682406,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":727,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743638+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:26.743638+0000","last_peered":"2024-09-21T15:34:26.743638+0000","last_clean":"2024-09-21T15:34:26.743638+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:26.743638+0000","last_undegraded":"2024-09-21T15:34:26.743638+0000","last_fullsized":"2024-09-21T15:34:26.743638+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"96'391","reported_seq":5666,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-21T15:34:28.072803+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:28.072803+0000","last_peered":"2024-09-21T15:34:28.072803+0000","last_clean":"2024-09-21T15:34:28.072803+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:28.072803+0000","last_undegraded":"2024-09-21T15:34:28.072803+0000","last_fullsized":"2024-09-21T15:34:28.072803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":25206458,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":7269,"num_read_kb":18206,"num_write":524,"num_write_kb":17322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"96'456","reported_seq":5535,"reported_epoch":96,"state":"active+undersized+remapped","last_fresh":"2024-09-21T15:34:28.228586+0000","last_change":"2024-09-21T15:34:28.228848+0000","last_active":"2024-09-21T15:34:28.228586+0000","last_peered":"2024-09-21T15:34:28.228586+0000","last_clean":"2024-09-21T15:34:26.155415+0000","last_became_active":"2024-09-21T15:34:27.760062+0000","last_became_peered":"2024-09-21T15:34:27.760062+0000","last_unstale":"2024-09-21T15:34:28.228586+0000","last_undegraded":"2024-09-21T15:34:27.741167+0000","last_fullsized":"2024-09-21T15:34:27.739664+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:47:48.096630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":19198544,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":12,"num_read":6568,"num_read_kb":6356,"num_write":604,"num_write_kb":2570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":1099128,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":["0","1","3"],"object_location_counts":[{"shards":"0","objects":42}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"93'311","reported_seq":1811,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-21T15:34:26.743906+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:26.743906+0000","last_peered":"2024-09-21T15:34:26.743906+0000","last_clean":"2024-09-21T15:34:26.743906+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:26.743906+0000","last_undegraded":"2024-09-21T15:34:26.743906+0000","last_fullsized":"2024-09-21T15:34:26.743906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":28644623,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":4098,"num_read_kb":5065,"num_write":300,"num_write_kb":10364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":831315617,"num_objects":1297,"num_object_clones":290,"num_object_copies":2594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1297,"num_whiteouts":464,"num_read":185546,"num_read_kb":293115,"num_write":13060,"num_write_kb":286268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":11685642,"num_keys_recovered":94,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167899136,"data_stored":203852805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10364,"ondisk_log_size":10364,"up":64,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53296,"kb_used_data":26272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318544,"statfs":{"total":96636764160,"available":96582189056,"internally_reserved":0,"allocated":26902528,"data_stored":30884872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.748}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.88},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.55500000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":11912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":12197888,"data_stored":13498496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.5},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.625}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":1.073},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.1120000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0029999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.198},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.78700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":24480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":25067520,"data_stored":29579967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.498}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64600000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61576,"kb_used_data":30464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94310264,"statfs":{"total":96636764160,"available":96573710336,"internally_reserved":0,"allocated":31195136,"data_stored":35482439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0389999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.016},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.82499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.0620000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.80800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75328,"kb_used_data":36012,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94296512,"statfs":{"total":96636764160,"available":96559628288,"internally_reserved":0,"allocated":36876288,"data_stored":45858618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.222}]},{"osd":1,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.316},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.36599999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.42999999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.51400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41996,"kb_used_data":6788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329844,"statfs":{"total":96636764160,"available":96593760256,"internally_reserved":0,"allocated":6950912,"data_stored":7366987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.91100000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":15812,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":16191488,"data_stored":18844083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51800000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.77200000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.03},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49296,"kb_used_data":22268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322544,"statfs":{"total":96636764160,"available":96586285056,"internally_reserved":0,"allocated":22802432,"data_stored":27712975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.96199999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0189999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.98999999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2649999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.371},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.038}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21069824,"data_stored":26595870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15048704,"data_stored":18317327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5812224,"data_stored":6840250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35737600,"data_stored":45331881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30056448,"data_stored":34955702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23924736,"data_stored":29053211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":12400429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25763840,"data_stored":30358135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:32.608 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:32.606+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:32.629 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:32.630 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:32.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:32.656 INFO:teuthology.orchestra.run.smithi107.stdout:4919 trunc from 0xe55fa70 to 0x27c7b96 2024-09-21T15:34:32.677 INFO:teuthology.orchestra.run.smithi107.stdout:4920 write 0x575251c thru 0x57604f8 (0xdfdd bytes) 2024-09-21T15:34:32.687 INFO:teuthology.orchestra.run.smithi107.stdout:4922 punch from 0x159a84b to 0x15a0bd1, (0x6386 bytes) 2024-09-21T15:34:32.687 INFO:teuthology.orchestra.run.smithi107.stdout:4923 read 0x30471c1 thru 0x304b2b7 (0x40f7 bytes) 2024-09-21T15:34:32.689 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:32.688+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:32.701 INFO:teuthology.orchestra.run.smithi107.stdout:4925 write 0xa9eb9dc thru 0xa9eff62 (0x4587 bytes) 2024-09-21T15:34:32.706 INFO:teuthology.orchestra.run.smithi107.stdout:4926 writesame 0x3beb693 thru 0x3bef85e data_size 0x2(0x41cc bytes) 2024-09-21T15:34:32.716 INFO:teuthology.orchestra.run.smithi107.stdout:4927 trunc from 0xa9eff63 to 0xb3cc2b8 2024-09-21T15:34:32.726 INFO:teuthology.orchestra.run.smithi107.stdout:4928 writesame 0x9b08167 thru 0x9b0cb5d data_size 0x7(0x49f7 bytes) 2024-09-21T15:34:32.732 INFO:teuthology.orchestra.run.smithi107.stdout:4929 read 0x7f4b263 thru 0x7f511c7 (0x5f65 bytes) 2024-09-21T15:34:32.732 INFO:teuthology.orchestra.run.smithi107.stdout:4931 write 0xf9f3a8 thru 0xfa6a38 (0x7691 bytes) 2024-09-21T15:34:32.739 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:32.739 INFO:teuthology.orchestra.run.smithi107.stdout:4933 write 0x92a19da thru 0x92aba81 (0xa0a8 bytes) 2024-09-21T15:34:32.757 INFO:teuthology.orchestra.run.smithi107.stdout:4935 write 0xd2a5bd2 thru 0xd2b3bb6 (0xdfe5 bytes) 2024-09-21T15:34:32.765 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:32.765 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:32.766 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:32.766 INFO:teuthology.orchestra.run.smithi107.stdout:4937 write 0x222a381 thru 0x2231e26 (0x7aa6 bytes) 2024-09-21T15:34:32.773 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:32.772+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:32.779 INFO:teuthology.orchestra.run.smithi107.stdout:4938 trunc from 0xd2b3bb7 to 0x9672d25 2024-09-21T15:34:32.781 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:32.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:34:32.801 INFO:teuthology.orchestra.run.smithi107.stdout:4940 punch from 0x86bd284 to 0x86c2301, (0x507d bytes) 2024-09-21T15:34:32.801 INFO:teuthology.orchestra.run.smithi107.stdout:4941 writesame 0x937a1bd thru 0x938808c data_size 0x3e(0xded0 bytes) 2024-09-21T15:34:32.877 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:32.876+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:32.966 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:33.009 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:33.008+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:33.075 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:33.070+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:33.133 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.133 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:33.196 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:33.194+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:33.268 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.268 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.276 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:33.274+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:33.281 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:33.378 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:33.376+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:33.441 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.441 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:33.500 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:33.498+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:33.568 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.568 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:33.580 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:33.578+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:33.681 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:33.680+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:33.743 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.743 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:33.793 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:34:33.794 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:33.790+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:33.803 INFO:teuthology.orchestra.run.smithi107.stdout:4942 write 0x851fce thru 0x859b11 (0x7b44 bytes) 2024-09-21T15:34:33.827 INFO:teuthology.orchestra.run.smithi107.stdout:4943 writesame 0xe046d44 thru 0xe052c69 data_size 0x2b(0xbf26 bytes) 2024-09-21T15:34:33.828 INFO:teuthology.orchestra.run.smithi107.stdout:4944 write 0x926da3a thru 0x9270426 (0x29ed bytes) 2024-09-21T15:34:33.834 INFO:teuthology.orchestra.run.smithi107.stdout:4945 punch from 0x9f8cf89 to 0x9f9cdeb, (0xfe62 bytes) 2024-09-21T15:34:33.834 INFO:teuthology.orchestra.run.smithi107.stdout:4946 writesame 0x75f0117 thru 0x75fea8d data_size 0x1(0xe977 bytes) 2024-09-21T15:34:33.872 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:33.872 INFO:teuthology.orchestra.run.smithi107.stdout:4948 read 0x80398ab thru 0x804553c (0xbc92 bytes) 2024-09-21T15:34:33.873 INFO:teuthology.orchestra.run.smithi107.stdout:4950 clone 37 order 21 su 1048576 sc 7 2024-09-21T15:34:33.883 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:33.880+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:33.886 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:33.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:33.984 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:33.980+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:34.057 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:34.057 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:34.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:34.120 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:34.118+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:34.185 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:34.184+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:34.189 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:34.189 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:34.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:34.292 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:34.290+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:34.370 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:34.370 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:34.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:34.428 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:34.426+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:34.487 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:34.484+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:34.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:34.550 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:34.550 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:34.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:34.612 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:34.610+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:34.687 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:34.688 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:34.686+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:34.703 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:34.719 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:34.726 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:34.727 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:34.735 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:34.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:34.755 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:34.756 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:34.775 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:34.776 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:34.789 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:34.788+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:34.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:34.797 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:34.820 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:34.820 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:34.877 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:34.877 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:34.877 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:34.889 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:34.886+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:34.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:34.990 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:34.986+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.692421822, 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.061 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:35.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.664210646, 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.070 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 101.47491127799999, 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.071 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:35.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 101.343526587, 2024-09-21T15:34:35.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:35.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.080 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.182429182999996, 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.182376636000001, 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:35.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:35.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:35.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.182328510999994, 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:35.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:35.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:35.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 98.169931622999997, 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:35.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.609158640999993, 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:35.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:35.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.542134274999995, 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.109 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.407014144000001, 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:35.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:35.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:35.120 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.366352000999996, 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.124 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.366338784000007, 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.366325977000002, 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.129 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:35.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.366285570000002, 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:35.137 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.366261101999996, 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.141 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.347039113999998, 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:35.146 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.259735531000004, 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:35.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:35.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:35.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 28.265050087999999, 2024-09-21T15:34:35.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:35.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.156 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 27.850790741000001, 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:35.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:35.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:35.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:35.160 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:35.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:35.162 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T15:34:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:4951 punch from 0x85f4584 to 0x85f49a8, (0x424 bytes) 2024-09-21T15:34:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:4953 read 0x3697b5d thru 0x3699bc9 (0x206d bytes) 2024-09-21T15:34:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:4956 write 0xc3f471c thru 0xc3f772f (0x3014 bytes) 2024-09-21T15:34:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:4957 trunc from 0xe052c6a to 0x11e7ae7 2024-09-21T15:34:35.163 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:35.160+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:35.185 INFO:teuthology.orchestra.run.smithi107.stdout:4958 write 0x2e99fe0 thru 0x2ea829a (0xe2bb bytes) 2024-09-21T15:34:35.202 INFO:teuthology.orchestra.run.smithi107.stdout:4959 write 0x3a3383f thru 0x3a4185f (0xe021 bytes) 2024-09-21T15:34:35.227 INFO:teuthology.orchestra.run.smithi107.stdout:4960 write 0x7c6046d thru 0x7c6d96c (0xd500 bytes) 2024-09-21T15:34:35.233 INFO:teuthology.orchestra.run.smithi107.stdout:4961 read 0x4b84bf9 thru 0x4b8de2d (0x9235 bytes) 2024-09-21T15:34:35.252 INFO:teuthology.orchestra.run.smithi107.stdout:4962 write 0xb4ffeef thru 0xb505032 (0x5144 bytes) 2024-09-21T15:34:35.262 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.262 INFO:teuthology.orchestra.run.smithi107.stdout:4965 punch from 0x65c1d4b to 0x65cbb11, (0x9dc6 bytes) 2024-09-21T15:34:35.262 INFO:teuthology.orchestra.run.smithi107.stdout:4966 read 0x22757ba thru 0x227a05f (0x48a6 bytes) 2024-09-21T15:34:35.262 INFO:teuthology.orchestra.run.smithi107.stdout:4967 read 0x1adba4f thru 0x1aea0d6 (0xe688 bytes) 2024-09-21T15:34:35.262 INFO:teuthology.orchestra.run.smithi107.stdout:4969 read 0x302de27 thru 0x303764b (0x9825 bytes) 2024-09-21T15:34:35.262 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:35.260+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:35.263 INFO:teuthology.orchestra.run.smithi107.stdout:4970 writesame 0x2216572 thru 0x2224672 data_size 0x1(0xe101 bytes) 2024-09-21T15:34:35.297 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:35.297 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:35.297 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:35.297 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:35.311 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:35.341 INFO:teuthology.orchestra.run.smithi107.stdout:4973 write 0x3890e39 thru 0x389a45b (0x9623 bytes) 2024-09-21T15:34:35.349 INFO:teuthology.orchestra.run.smithi107.stdout:4974 punch from 0x1dd129b to 0x1dd8229, (0x6f8e bytes) 2024-09-21T15:34:35.350 INFO:teuthology.orchestra.run.smithi107.stdout:4975 punch from 0x7cfdbe3 to 0x7d03387, (0x57a4 bytes) 2024-09-21T15:34:35.350 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.350 INFO:teuthology.orchestra.run.smithi107.stdout:4979 write 0x6d5498f thru 0x6d64194 (0xf806 bytes) 2024-09-21T15:34:35.357 INFO:teuthology.orchestra.run.smithi107.stdout:4980 read 0xafaf109 thru 0xafb6a99 (0x7991 bytes) 2024-09-21T15:34:35.357 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.357 INFO:teuthology.orchestra.run.smithi107.stdout:4985 read 0x3bad49 thru 0x3c4e9e (0xa156 bytes) 2024-09-21T15:34:35.379 INFO:teuthology.orchestra.run.smithi107.stdout:4986 writesame 0xe864c9a thru 0xe8746dc data_size 0x1(0xfa43 bytes) 2024-09-21T15:34:35.379 INFO:teuthology.orchestra.run.smithi107.stdout:4987 write 0x65cce1 thru 0x66cc0a (0xff2a bytes) 2024-09-21T15:34:35.388 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.388 INFO:teuthology.orchestra.run.smithi107.stdout:4989 writesame 0x26ec0b thru 0x27b03d data_size 0x1(0xc433 bytes) 2024-09-21T15:34:35.397 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:35.396+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:35.434 INFO:teuthology.orchestra.run.smithi107.stdout:4990 write 0xa5aa9f thru 0xa5de48 (0x33aa bytes) 2024-09-21T15:34:35.443 INFO:teuthology.orchestra.run.smithi107.stdout:4992 read 0x17e2e6c thru 0x17e614f (0x32e4 bytes) 2024-09-21T15:34:35.444 INFO:teuthology.orchestra.run.smithi107.stdout:4993 punch from 0xb740476 to 0xb74fac7, (0xf651 bytes) 2024-09-21T15:34:35.444 INFO:teuthology.orchestra.run.smithi107.stdout:4994 write 0xd059b1b thru 0xd063fb4 (0xa49a bytes) 2024-09-21T15:34:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:4996 read 0x8a51d83 thru 0x8a5481d (0x2a9b bytes) 2024-09-21T15:34:35.463 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:35.460+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:35.475 INFO:teuthology.orchestra.run.smithi107.stdout:4997 trunc from 0xe8746dd to 0x5ee3794 2024-09-21T15:34:35.479 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:35.479 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:35.479 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:35.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:35.496 INFO:teuthology.orchestra.run.smithi107.stdout:4999 read 0x394e68d thru 0x3958ea8 (0xa81c bytes) 2024-09-21T15:34:35.496 INFO:teuthology.orchestra.run.smithi107.stdout:5001 read 0x4118fc8 thru 0x411ca2c (0x3a65 bytes) 2024-09-21T15:34:35.497 INFO:teuthology.orchestra.run.smithi107.stdout:5003 writesame 0x548e1e6 thru 0x5494016 data_size 0x1(0x5e31 bytes) 2024-09-21T15:34:35.557 INFO:teuthology.orchestra.run.smithi107.stdout:5005 writesame 0x9f71b74 thru 0x9f74d4d data_size 0x9(0x31da bytes) 2024-09-21T15:34:35.564 INFO:teuthology.orchestra.run.smithi107.stdout:5007 trunc from 0x9f74d4e to 0x6fc0a1a 2024-09-21T15:34:35.564 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:35.562+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:35.569 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:35.574 INFO:teuthology.orchestra.run.smithi107.stdout:5008 read 0xc131bc thru 0xc20d51 (0xdb96 bytes) 2024-09-21T15:34:35.574 INFO:teuthology.orchestra.run.smithi107.stdout:5009 writesame 0x46e30ea thru 0x46e83fb data_size 0x3e(0x5312 bytes) 2024-09-21T15:34:35.596 INFO:teuthology.orchestra.run.smithi107.stdout:5010 write 0x9b04ad6 thru 0x9b075ab (0x2ad6 bytes) 2024-09-21T15:34:35.615 INFO:teuthology.orchestra.run.smithi107.stdout:5011 write 0xbc70e79 thru 0xbc7e6b1 (0xd839 bytes) 2024-09-21T15:34:35.628 INFO:teuthology.orchestra.run.smithi107.stdout:5012 write 0xc1a25fe thru 0xc1a5f6d (0x3970 bytes) 2024-09-21T15:34:35.633 INFO:teuthology.orchestra.run.smithi107.stdout:5013 read 0x5b75839 thru 0x5b78fa3 (0x376b bytes) 2024-09-21T15:34:35.655 INFO:teuthology.orchestra.run.smithi107.stdout:5014 trunc from 0xc1a5f6e to 0x2b07b66 2024-09-21T15:34:35.665 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:35.662+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395194584999999, 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.675 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:35.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:35.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395170526000001, 2024-09-21T15:34:35.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.683 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395146563000001, 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:35.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:35.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395122796999999, 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:35.691 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395099155, 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.692 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:35.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.695 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395073602, 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.696 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:35.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:35.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:35.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.700 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395029387000001, 2024-09-21T15:34:35.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:35.705 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.395000677999999, 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:35.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:35.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.394969135, 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:35.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:35.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.713 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.394942062999998, 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:35.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:35.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:35.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.394916129999999, 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.721 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.394381187, 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:35.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.393751518999998, 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389839955999999, 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.730 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:35.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:35.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:35.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389793458, 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:35.734 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389744288999999, 2024-09-21T15:34:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.742 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389696441000002, 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:35.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:35.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389648051999998, 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:35.750 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.389567131, 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.751 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:35.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.385968768000001, 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:35.755 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:35.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:35.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:35.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:35.759 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5015 write 0x9d8e103 thru 0x9d988fd (0xa7fb bytes) 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5018 read 0x1e20021 thru 0x1e2d19f (0xd17f bytes) 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5019 trunc from 0x9d988fe to 0x9f37163 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5020 read 0x8b76ff9 thru 0x8b80f12 (0x9f1a bytes) 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5021 trunc from 0x9f37163 to 0x1ac4760 2024-09-21T15:34:35.761 INFO:teuthology.orchestra.run.smithi107.stdout:5022 punch from 0x11947d7 to 0x119543c, (0xc65 bytes) 2024-09-21T15:34:35.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:35.771 INFO:teuthology.orchestra.run.smithi107.stdout:5023 write 0xd738019 thru 0xd73c316 (0x42fe bytes) 2024-09-21T15:34:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:5024 read 0xa455e32 thru 0xa4651ef (0xf3be bytes) 2024-09-21T15:34:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:5025 punch from 0xd0608ea to 0xd06b53f, (0xac55 bytes) 2024-09-21T15:34:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:5028 read 0xc14809c thru 0xc14e6b5 (0x661a bytes) 2024-09-21T15:34:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:5029 write 0xc43cd5d thru 0xc447882 (0xab26 bytes) 2024-09-21T15:34:35.779 INFO:teuthology.orchestra.run.smithi107.stdout:5030 read 0x5cdc959 thru 0x5cea0e8 (0xd790 bytes) 2024-09-21T15:34:35.779 INFO:teuthology.orchestra.run.smithi107.stdout:5031 clone 38 order 25 su 16777216 sc 14 2024-09-21T15:34:35.813 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:35.812+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:35.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:34:35.867 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:35.864+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:35.891 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone37 from 0xd73c317 (overlap 0x11e7ae7) to 0xabe357 2024-09-21T15:34:35.900 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:35.900 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:35.900 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:35.901 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:35.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:35.907 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:35.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:35.928 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-21T15:34:35.807014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":857923979,"num_objects":1344,"num_object_clones":296,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":469,"num_read":197263,"num_read_kb":316747,"num_write":13488,"num_write_kb":296466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":20367099,"num_keys_recovered":127,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10687,"ondisk_log_size":10687,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456556,"kb_used_data":191224,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754518164,"statfs":{"total":773094113280,"available":772626599936,"internally_reserved":0,"allocated":195813376,"data_stored":227050989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":271568441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26017975,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":5,"num_read":11601,"num_read_kb":23532,"num_write":267,"num_write_kb":7780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":8091089,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033093"},"pg_stats":[{"pgid":"3.1f","version":"101'347","reported_seq":4839,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784224+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:32.784224+0000","last_peered":"2024-09-21T15:34:32.784224+0000","last_clean":"2024-09-21T15:34:32.784224+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:32.784224+0000","last_undegraded":"2024-09-21T15:34:32.784224+0000","last_fullsized":"2024-09-21T15:34:32.784224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5636,"num_read_kb":6344,"num_write":501,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'309","reported_seq":6353,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.787627+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:32.787627+0000","last_peered":"2024-09-21T15:34:32.787627+0000","last_clean":"2024-09-21T15:34:32.787627+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:32.787627+0000","last_undegraded":"2024-09-21T15:34:32.787627+0000","last_fullsized":"2024-09-21T15:34:32.787627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":31764946,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":7097,"num_read_kb":19426,"num_write":437,"num_write_kb":26063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"101'128","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.789733+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:32.789733+0000","last_peered":"2024-09-21T15:34:32.789733+0000","last_clean":"2024-09-21T15:34:32.789733+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:32.789733+0000","last_undegraded":"2024-09-21T15:34:32.789733+0000","last_fullsized":"2024-09-21T15:34:32.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":32630107,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":263,"num_read_kb":4854,"num_write":183,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"101'209","reported_seq":1342,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.789648+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:32.789648+0000","last_peered":"2024-09-21T15:34:32.789648+0000","last_clean":"2024-09-21T15:34:32.789648+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:32.789648+0000","last_undegraded":"2024-09-21T15:34:32.789648+0000","last_fullsized":"2024-09-21T15:34:32.789648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46862032,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1235,"num_read_kb":4875,"num_write":219,"num_write_kb":5001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"101'304","reported_seq":9167,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.785278+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:32.785278+0000","last_peered":"2024-09-21T15:34:32.785278+0000","last_clean":"2024-09-21T15:34:32.785278+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:32.785278+0000","last_undegraded":"2024-09-21T15:34:32.785278+0000","last_fullsized":"2024-09-21T15:34:32.785278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":17839064,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":9558,"num_read_kb":9255,"num_write":307,"num_write_kb":5274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"100'174","reported_seq":2297,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786806+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:32.786806+0000","last_peered":"2024-09-21T15:34:32.786806+0000","last_clean":"2024-09-21T15:34:32.786806+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:32.786806+0000","last_undegraded":"2024-09-21T15:34:32.786806+0000","last_fullsized":"2024-09-21T15:34:32.786806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":18286207,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":2123,"num_read_kb":11395,"num_write":173,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"100'551","reported_seq":33234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.739552+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:32.739552+0000","last_peered":"2024-09-21T15:34:32.739552+0000","last_clean":"2024-09-21T15:34:32.739552+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:32.739552+0000","last_undegraded":"2024-09-21T15:34:32.739552+0000","last_fullsized":"2024-09-21T15:34:32.739552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":63587805,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":34585,"num_read_kb":19435,"num_write":788,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"100'391","reported_seq":911,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.713116+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:32.713116+0000","last_peered":"2024-09-21T15:34:32.713116+0000","last_clean":"2024-09-21T15:34:32.713116+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:32.713116+0000","last_undegraded":"2024-09-21T15:34:32.713116+0000","last_fullsized":"2024-09-21T15:34:32.713116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":17567475,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":2042,"num_read_kb":4674,"num_write":552,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"100'578","reported_seq":1779,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.747714+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:32.747714+0000","last_peered":"2024-09-21T15:34:32.747714+0000","last_clean":"2024-09-21T15:34:32.747714+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:32.747714+0000","last_undegraded":"2024-09-21T15:34:32.747714+0000","last_fullsized":"2024-09-21T15:34:32.747714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":29991782,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3846,"num_read_kb":9634,"num_write":888,"num_write_kb":7466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'291","reported_seq":5759,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.828982+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:31.828982+0000","last_peered":"2024-09-21T15:34:31.828982+0000","last_clean":"2024-09-21T15:34:31.828982+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:31.828982+0000","last_undegraded":"2024-09-21T15:34:31.828982+0000","last_fullsized":"2024-09-21T15:34:31.828982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"100'232","reported_seq":4953,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-21T15:34:32.775097+0000","last_change":"2024-09-21T15:34:33.682238+0000","last_active":"2024-09-21T15:34:33.682238+0000","last_peered":"2024-09-21T15:34:32.775097+0000","last_clean":"2024-09-21T15:34:24.521559+0000","last_became_active":"2024-09-21T15:34:31.835981+0000","last_became_peered":"2024-09-21T15:34:31.835981+0000","last_unstale":"2024-09-21T15:34:32.775097+0000","last_undegraded":"2024-09-21T15:34:32.775097+0000","last_fullsized":"2024-09-21T15:34:31.779671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":11602304,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":4812,"num_read_kb":8549,"num_write":269,"num_write_kb":10345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"101'524","reported_seq":1657,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786192+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:32.786192+0000","last_peered":"2024-09-21T15:34:32.786192+0000","last_clean":"2024-09-21T15:34:32.786192+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:32.786192+0000","last_undegraded":"2024-09-21T15:34:32.786192+0000","last_fullsized":"2024-09-21T15:34:32.786192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":28098524,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":3555,"num_read_kb":6006,"num_write":671,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.780356+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:34:31.780356+0000","last_peered":"2024-09-21T15:34:31.780356+0000","last_clean":"2024-09-21T15:34:31.780356+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:34:31.780356+0000","last_undegraded":"2024-09-21T15:34:31.780356+0000","last_fullsized":"2024-09-21T15:34:31.780356+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'299","reported_seq":3527,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.632594+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:32.632594+0000","last_peered":"2024-09-21T15:34:32.632594+0000","last_clean":"2024-09-21T15:34:32.632594+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:32.632594+0000","last_undegraded":"2024-09-21T15:34:32.632594+0000","last_fullsized":"2024-09-21T15:34:32.632594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":5364,"num_read_kb":10540,"num_write":279,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.783586+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:31.783586+0000","last_peered":"2024-09-21T15:34:31.783586+0000","last_clean":"2024-09-21T15:34:31.783586+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:31.783586+0000","last_undegraded":"2024-09-21T15:34:31.783586+0000","last_fullsized":"2024-09-21T15:34:31.783586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"101'181","reported_seq":1198,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784239+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:32.784239+0000","last_peered":"2024-09-21T15:34:32.784239+0000","last_clean":"2024-09-21T15:34:32.784239+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:32.784239+0000","last_undegraded":"2024-09-21T15:34:32.784239+0000","last_fullsized":"2024-09-21T15:34:32.784239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":37229564,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":1378,"num_read_kb":21000,"num_write":201,"num_write_kb":18044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.834198+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:31.834198+0000","last_peered":"2024-09-21T15:34:31.834198+0000","last_clean":"2024-09-21T15:34:31.834198+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:31.834198+0000","last_undegraded":"2024-09-21T15:34:31.834198+0000","last_fullsized":"2024-09-21T15:34:31.834198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"101'169","reported_seq":719,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.788333+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:32.788333+0000","last_peered":"2024-09-21T15:34:32.788333+0000","last_clean":"2024-09-21T15:34:32.788333+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:32.788333+0000","last_undegraded":"2024-09-21T15:34:32.788333+0000","last_fullsized":"2024-09-21T15:34:32.788333+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":16556525,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":806,"num_read_kb":4007,"num_write":174,"num_write_kb":2721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.783445+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:31.783445+0000","last_peered":"2024-09-21T15:34:31.783445+0000","last_clean":"2024-09-21T15:34:31.783445+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:31.783445+0000","last_undegraded":"2024-09-21T15:34:31.783445+0000","last_fullsized":"2024-09-21T15:34:31.783445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'376","reported_seq":769,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829545+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:31.829545+0000","last_peered":"2024-09-21T15:34:31.829545+0000","last_clean":"2024-09-21T15:34:31.829545+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:31.829545+0000","last_undegraded":"2024-09-21T15:34:31.829545+0000","last_fullsized":"2024-09-21T15:34:31.829545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":12612201,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1449,"num_read_kb":4766,"num_write":486,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"101'187","reported_seq":5997,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786988+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:32.786988+0000","last_peered":"2024-09-21T15:34:32.786988+0000","last_clean":"2024-09-21T15:34:32.786988+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:32.786988+0000","last_undegraded":"2024-09-21T15:34:32.786988+0000","last_fullsized":"2024-09-21T15:34:32.786988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":23544973,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":5793,"num_read_kb":11966,"num_write":212,"num_write_kb":4008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":98,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829722+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:31.829722+0000","last_peered":"2024-09-21T15:34:31.829722+0000","last_clean":"2024-09-21T15:34:31.829722+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:31.829722+0000","last_undegraded":"2024-09-21T15:34:31.829722+0000","last_fullsized":"2024-09-21T15:34:31.829722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'669","reported_seq":1799,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.808154+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:32.808154+0000","last_peered":"2024-09-21T15:34:32.808154+0000","last_clean":"2024-09-21T15:34:32.808154+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:32.808154+0000","last_undegraded":"2024-09-21T15:34:32.808154+0000","last_fullsized":"2024-09-21T15:34:32.808154+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25820728,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":4347,"num_read_kb":5726,"num_write":952,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.781274+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:31.781274+0000","last_peered":"2024-09-21T15:34:31.781274+0000","last_clean":"2024-09-21T15:34:31.781274+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:31.781274+0000","last_undegraded":"2024-09-21T15:34:31.781274+0000","last_fullsized":"2024-09-21T15:34:31.781274+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'426","reported_seq":2461,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.801306+0000","last_change":"2024-09-21T15:34:29.764833+0000","last_active":"2024-09-21T15:34:32.801306+0000","last_peered":"2024-09-21T15:34:32.801306+0000","last_clean":"2024-09-21T15:34:32.801306+0000","last_became_active":"2024-09-21T15:34:29.762828+0000","last_became_peered":"2024-09-21T15:34:29.762828+0000","last_unstale":"2024-09-21T15:34:32.801306+0000","last_undegraded":"2024-09-21T15:34:32.801306+0000","last_fullsized":"2024-09-21T15:34:32.801306+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:17.503095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":24984429,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":3335,"num_read_kb":8883,"num_write":625,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.831473+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:31.831473+0000","last_peered":"2024-09-21T15:34:31.831473+0000","last_clean":"2024-09-21T15:34:31.831473+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:31.831473+0000","last_undegraded":"2024-09-21T15:34:31.831473+0000","last_fullsized":"2024-09-21T15:34:31.831473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'203","reported_seq":705,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.966656+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:31.966656+0000","last_peered":"2024-09-21T15:34:31.966656+0000","last_clean":"2024-09-21T15:34:31.966656+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:31.966656+0000","last_undegraded":"2024-09-21T15:34:31.966656+0000","last_fullsized":"2024-09-21T15:34:31.966656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":23697932,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":991,"num_read_kb":3966,"num_write":190,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":121,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829457+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:31.829457+0000","last_peered":"2024-09-21T15:34:31.829457+0000","last_clean":"2024-09-21T15:34:31.829457+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:31.829457+0000","last_undegraded":"2024-09-21T15:34:31.829457+0000","last_fullsized":"2024-09-21T15:34:31.829457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":101,"state":"peering","last_fresh":"2024-09-21T15:34:31.845196+0000","last_change":"2024-09-21T15:34:33.682082+0000","last_active":"2024-09-21T15:34:33.682082+0000","last_peered":"2024-09-21T15:34:31.845196+0000","last_clean":"2024-09-21T15:34:28.744238+0000","last_became_active":"2024-09-21T15:34:31.832616+0000","last_became_peered":"2024-09-21T15:34:31.832616+0000","last_unstale":"2024-09-21T15:34:31.845196+0000","last_undegraded":"2024-09-21T15:34:31.845196+0000","last_fullsized":"2024-09-21T15:34:31.777222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:57.645467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"100'474","reported_seq":1391,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.688193+0000","last_change":"2024-09-21T15:34:29.765868+0000","last_active":"2024-09-21T15:34:32.688193+0000","last_peered":"2024-09-21T15:34:32.688193+0000","last_clean":"2024-09-21T15:34:32.688193+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:34:32.688193+0000","last_undegraded":"2024-09-21T15:34:32.688193+0000","last_fullsized":"2024-09-21T15:34:32.688193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":12931068,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2587,"num_read_kb":6802,"num_write":662,"num_write_kb":5716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'454","reported_seq":1505,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.663535+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:32.663535+0000","last_peered":"2024-09-21T15:34:32.663535+0000","last_clean":"2024-09-21T15:34:32.663535+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:32.663535+0000","last_undegraded":"2024-09-21T15:34:32.663535+0000","last_fullsized":"2024-09-21T15:34:32.663535+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522187,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":3121,"num_read_kb":7563,"num_write":615,"num_write_kb":3711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"101'263","reported_seq":4752,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.787511+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:32.787511+0000","last_peered":"2024-09-21T15:34:32.787511+0000","last_clean":"2024-09-21T15:34:32.787511+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:32.787511+0000","last_undegraded":"2024-09-21T15:34:32.787511+0000","last_fullsized":"2024-09-21T15:34:32.787511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":21750935,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4749,"num_read_kb":13714,"num_write":287,"num_write_kb":5517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"100'297","reported_seq":4678,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.636499+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:32.636499+0000","last_peered":"2024-09-21T15:34:32.636499+0000","last_clean":"2024-09-21T15:34:32.636499+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:32.636499+0000","last_undegraded":"2024-09-21T15:34:32.636499+0000","last_fullsized":"2024-09-21T15:34:32.636499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":6030,"num_read_kb":11328,"num_write":324,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"101'221","reported_seq":1956,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.790627+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:32.790627+0000","last_peered":"2024-09-21T15:34:32.790627+0000","last_clean":"2024-09-21T15:34:32.790627+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:32.790627+0000","last_undegraded":"2024-09-21T15:34:32.790627+0000","last_fullsized":"2024-09-21T15:34:32.790627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":35350897,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":1818,"num_read_kb":8299,"num_write":229,"num_write_kb":21213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17295,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.834220+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:31.834220+0000","last_peered":"2024-09-21T15:34:31.834220+0000","last_clean":"2024-09-21T15:34:31.834220+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:31.834220+0000","last_undegraded":"2024-09-21T15:34:31.834220+0000","last_fullsized":"2024-09-21T15:34:31.834220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17967,"num_read_kb":4981,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"101'283","reported_seq":33608,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784034+0000","last_change":"2024-09-21T15:34:30.767437+0000","last_active":"2024-09-21T15:34:32.784034+0000","last_peered":"2024-09-21T15:34:32.784034+0000","last_clean":"2024-09-21T15:34:32.784034+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:34:32.784034+0000","last_undegraded":"2024-09-21T15:34:32.784034+0000","last_fullsized":"2024-09-21T15:34:32.784034+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":33828,"num_read_kb":32899,"num_write":318,"num_write_kb":20815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"100'303","reported_seq":1574,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-21T15:34:32.685973+0000","last_change":"2024-09-21T15:34:33.682386+0000","last_active":"2024-09-21T15:34:33.682386+0000","last_peered":"2024-09-21T15:34:32.685973+0000","last_clean":"2024-09-21T15:34:25.286842+0000","last_became_active":"2024-09-21T15:34:31.838100+0000","last_became_peered":"2024-09-21T15:34:31.838100+0000","last_unstale":"2024-09-21T15:34:32.685973+0000","last_undegraded":"2024-09-21T15:34:32.685973+0000","last_fullsized":"2024-09-21T15:34:32.685973+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":30043907,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":2337,"num_read_kb":6342,"num_write":346,"num_write_kb":3654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":732,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.831304+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:31.831304+0000","last_peered":"2024-09-21T15:34:31.831304+0000","last_clean":"2024-09-21T15:34:31.831304+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:31.831304+0000","last_undegraded":"2024-09-21T15:34:31.831304+0000","last_fullsized":"2024-09-21T15:34:31.831304+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"100'397","reported_seq":5729,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.087367+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:32.087367+0000","last_peered":"2024-09-21T15:34:32.087367+0000","last_clean":"2024-09-21T15:34:32.087367+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:32.087367+0000","last_undegraded":"2024-09-21T15:34:32.087367+0000","last_fullsized":"2024-09-21T15:34:32.087367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":26255034,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":11,"num_read":7440,"num_read_kb":18349,"num_write":533,"num_write_kb":18015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"100'459","reported_seq":7114,"reported_epoch":101,"state":"peering","last_fresh":"2024-09-21T15:34:32.785184+0000","last_change":"2024-09-21T15:34:32.785387+0000","last_active":"2024-09-21T15:34:32.785184+0000","last_peered":"2024-09-21T15:34:32.607757+0000","last_clean":"2024-09-21T15:34:26.155415+0000","last_became_active":"2024-09-21T15:34:31.842645+0000","last_became_peered":"2024-09-21T15:34:31.842645+0000","last_unstale":"2024-09-21T15:34:32.785184+0000","last_undegraded":"2024-09-21T15:34:32.785184+0000","last_fullsized":"2024-09-21T15:34:32.785184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:15.523620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":20247120,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":7026,"num_read_kb":10500,"num_write":608,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2138377,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"100'312","reported_seq":1818,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.774298+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:32.774298+0000","last_peered":"2024-09-21T15:34:32.774298+0000","last_clean":"2024-09-21T15:34:32.774298+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:32.774298+0000","last_undegraded":"2024-09-21T15:34:32.774298+0000","last_fullsized":"2024-09-21T15:34:32.774298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":29086991,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":4099,"num_read_kb":5101,"num_write":301,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":857333592,"num_objects":1340,"num_object_clones":296,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":469,"num_read":197147,"num_read_kb":316647,"num_write":13327,"num_write_kb":294048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":19186363,"num_keys_recovered":127,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184414208,"data_stored":220814343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10567,"ondisk_log_size":10567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63848,"kb_used_data":32732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94307992,"statfs":{"total":96636764160,"available":96571383808,"internally_reserved":0,"allocated":33517568,"data_stored":37973447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.77600000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.86899999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.98399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":1.1359999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.181},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.095}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":12640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":12943360,"data_stored":14264340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.501}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.82799999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.98199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.871},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":1.0589999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.036},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":1.0089999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":24016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":24592384,"data_stored":28961867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":1.0269999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.226},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.194}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.26},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1679999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.411},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.29}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67220,"kb_used_data":36100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94304620,"statfs":{"total":96636764160,"available":96567930880,"internally_reserved":0,"allocated":36966400,"data_stored":41423229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.96099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.627},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.67700000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.75800000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.72499999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.01}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.82999999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":1.046},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77420,"kb_used_data":38108,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94294420,"statfs":{"total":96636764160,"available":96557486080,"internally_reserved":0,"allocated":39022592,"data_stored":48012790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.624},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.499}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.85399999999999998},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":0.745}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42116,"kb_used_data":6908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329724,"statfs":{"total":96636764160,"available":96593637376,"internally_reserved":0,"allocated":7073792,"data_stored":7448762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.62}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.375},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0509999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.83999999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.90500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53784,"kb_used_data":14476,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94318056,"statfs":{"total":96636764160,"available":96581689344,"internally_reserved":0,"allocated":14823424,"data_stored":17225723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.626}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.434},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.745},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53268,"kb_used_data":26244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318572,"statfs":{"total":96636764160,"available":96582217728,"internally_reserved":0,"allocated":26873856,"data_stored":31740831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.41699999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.91100000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.95799999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25079808,"data_stored":30593335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13619200,"data_stored":16668576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":6891634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37822464,"data_stored":47455662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35766272,"data_stored":40866101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23388160,"data_stored":28404720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11149312,"data_stored":13116844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":31715328,"data_stored":36817471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:35.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T15:34:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T15:34:36.005 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:36.004+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:36.069 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:36.066+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:36.085 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:36.086 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:36.086 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:36.086 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:36.086 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:36.100 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:36.171 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:36.166+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:36.261 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.979311313, 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.979252752000001, 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.979182037000001, 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:34:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.979132402000001, 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:34:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.979064634, 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:34:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:34:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978953761, 2024-09-21T15:34:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978859794000002, 2024-09-21T15:34:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:34:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978811668999999, 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:34:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:34:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978741991, 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:34:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:34:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978692598999999, 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.97864573, 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978598635000001, 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:34:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:34:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978549597000001, 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:34:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:34:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978481849000001, 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:34:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:34:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:34:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978433169999999, 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.978322887000001, 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.976074430000001, 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.976004678999999, 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.975915027999999, 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.336 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:34:36.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:34:36.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.975635246, 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.340 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:34:36.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:34:36.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.344 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:36.346 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:36.346 INFO:teuthology.orchestra.run.smithi107.stdout:{"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726932748,"num_in_osds":8,"osd_in_since":1726932800,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"peering","count":2},{"state_name":"creating+peering","count":2}],"num_pgs":41,"num_pools":3,"num_objects":1344,"data_bytes":857923979,"bytes_used":467513344,"bytes_avail":772626599936,"bytes_total":773094113280,"inactive_pgs_ratio":0.097560971975326538,"recovering_objects_per_sec":70,"recovering_bytes_per_sec":2022166,"recovering_keys_per_sec":8,"num_objects_recovered":281,"num_bytes_recovered":8091089,"num_keys_recovered":33,"read_bytes_sec":6022388,"write_bytes_sec":1991083,"read_op_per_sec":2899,"write_op_per_sec":66},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-21T15:33:29.652795+0000","services":{}},"progress_events":{}} 2024-09-21T15:34:36.346 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:36.347 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:36.344+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:36.347 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T15:34:36.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:36.369 INFO:teuthology.orchestra.run.smithi107.stdout:5032 read 0x8d625cd thru 0x8d66734 (0x4168 bytes) 2024-09-21T15:34:36.370 INFO:teuthology.orchestra.run.smithi107.stdout:5033 writesame 0xd1d709f thru 0xd1e4fa7 data_size 0x1(0xdf09 bytes) 2024-09-21T15:34:36.443 INFO:teuthology.orchestra.run.smithi107.stdout:5034 read 0x9b99c20 thru 0x9b9eaed (0x4ece bytes) 2024-09-21T15:34:36.443 INFO:teuthology.orchestra.run.smithi107.stdout:5035 punch from 0x234be79 to 0x235262c, (0x67b3 bytes) 2024-09-21T15:34:36.443 INFO:teuthology.orchestra.run.smithi107.stdout:5036 write 0x4142bfd thru 0x414f9df (0xcde3 bytes) 2024-09-21T15:34:36.446 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:36.444+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:36.450 INFO:teuthology.orchestra.run.smithi107.stdout:5037 read 0x4b9f762 thru 0x4badd85 (0xe624 bytes) 2024-09-21T15:34:36.451 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.451 INFO:teuthology.orchestra.run.smithi107.stdout:5040 read 0xc5287a1 thru 0xc52892a (0x18a bytes) 2024-09-21T15:34:36.451 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.466 INFO:teuthology.orchestra.run.smithi107.stdout:5042 trunc from 0xd73c317 to 0x446c942 2024-09-21T15:34:36.512 INFO:teuthology.orchestra.run.smithi107.stdout:5045 writesame 0xe0eda27 thru 0xe0f9722 data_size 0x52(0xbcfc bytes) 2024-09-21T15:34:36.535 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:36.535 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:36.547 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:36.544+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:36.557 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:36.558 INFO:teuthology.orchestra.run.smithi107.stdout:5046 writesame 0x4715291 thru 0x47237b3 data_size 0x1(0xe523 bytes) 2024-09-21T15:34:36.635 INFO:teuthology.orchestra.run.smithi107.stdout:5047 writesame 0xed45466 thru 0xed45ebf data_size 0x5(0xa5a bytes) 2024-09-21T15:34:36.637 INFO:teuthology.orchestra.run.smithi107.stdout:5048 write 0x536c909 thru 0x53776b8 (0xadb0 bytes) 2024-09-21T15:34:36.642 INFO:teuthology.orchestra.run.smithi107.stdout:5049 read 0xd1ce390 thru 0xd1d9b8b (0xb7fc bytes) 2024-09-21T15:34:36.643 INFO:teuthology.orchestra.run.smithi107.stdout:5052 write 0xc077d09 thru 0xc082088 (0xa380 bytes) 2024-09-21T15:34:36.647 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:36.642+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:36.648 INFO:teuthology.orchestra.run.smithi107.stdout:5055 write 0x30049e6 thru 0x30138dd (0xeef8 bytes) 2024-09-21T15:34:36.654 INFO:teuthology.orchestra.run.smithi107.stdout:5057 write 0xde8b5d7 thru 0xde8c779 (0x11a3 bytes) 2024-09-21T15:34:36.659 INFO:teuthology.orchestra.run.smithi107.stdout:5058 punch from 0x71f0c95 to 0x71fe827, (0xdb92 bytes) 2024-09-21T15:34:36.659 INFO:teuthology.orchestra.run.smithi107.stdout:5059 write 0xb49c828 thru 0xb4a6dba (0xa593 bytes) 2024-09-21T15:34:36.665 INFO:teuthology.orchestra.run.smithi107.stdout:5060 read 0x6817db thru 0x684913 (0x3139 bytes) 2024-09-21T15:34:36.665 INFO:teuthology.orchestra.run.smithi107.stdout:5061 writesame 0x50f56ba thru 0x50ffba3 data_size 0x26(0xa4ea bytes) 2024-09-21T15:34:36.668 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.668 INFO:teuthology.orchestra.run.smithi107.stdout:5063 writesame 0xc0c63f3 thru 0xc0d104f data_size 0x19(0xac5d bytes) 2024-09-21T15:34:36.690 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:36.691 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:36.701 INFO:teuthology.orchestra.run.smithi107.stdout:5064 trunc from 0xed45ec0 to 0x891216 2024-09-21T15:34:36.717 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-21T15:34:35.807014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":857923979,"num_objects":1344,"num_object_clones":296,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":469,"num_read":197263,"num_read_kb":316747,"num_write":13488,"num_write_kb":296466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":20367099,"num_keys_recovered":127,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10687,"ondisk_log_size":10687,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456556,"kb_used_data":191224,"kb_used_omap":12,"kb_used_meta":265203,"kb_avail":754518164,"statfs":{"total":773094113280,"available":772626599936,"internally_reserved":0,"allocated":195813376,"data_stored":227050989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":271568441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26017975,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":5,"num_read":11601,"num_read_kb":23532,"num_write":267,"num_write_kb":7780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":8091089,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033093"},"pg_stats":[{"pgid":"3.1f","version":"101'347","reported_seq":4839,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784224+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:32.784224+0000","last_peered":"2024-09-21T15:34:32.784224+0000","last_clean":"2024-09-21T15:34:32.784224+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:32.784224+0000","last_undegraded":"2024-09-21T15:34:32.784224+0000","last_fullsized":"2024-09-21T15:34:32.784224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":5636,"num_read_kb":6344,"num_write":501,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'309","reported_seq":6353,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.787627+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:32.787627+0000","last_peered":"2024-09-21T15:34:32.787627+0000","last_clean":"2024-09-21T15:34:32.787627+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:32.787627+0000","last_undegraded":"2024-09-21T15:34:32.787627+0000","last_fullsized":"2024-09-21T15:34:32.787627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":31764946,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":7097,"num_read_kb":19426,"num_write":437,"num_write_kb":26063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"101'128","reported_seq":427,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.789733+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:32.789733+0000","last_peered":"2024-09-21T15:34:32.789733+0000","last_clean":"2024-09-21T15:34:32.789733+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:32.789733+0000","last_undegraded":"2024-09-21T15:34:32.789733+0000","last_fullsized":"2024-09-21T15:34:32.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":32630107,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":263,"num_read_kb":4854,"num_write":183,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"101'209","reported_seq":1342,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.789648+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:32.789648+0000","last_peered":"2024-09-21T15:34:32.789648+0000","last_clean":"2024-09-21T15:34:32.789648+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:32.789648+0000","last_undegraded":"2024-09-21T15:34:32.789648+0000","last_fullsized":"2024-09-21T15:34:32.789648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":46862032,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":1235,"num_read_kb":4875,"num_write":219,"num_write_kb":5001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"101'304","reported_seq":9167,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.785278+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:32.785278+0000","last_peered":"2024-09-21T15:34:32.785278+0000","last_clean":"2024-09-21T15:34:32.785278+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:32.785278+0000","last_undegraded":"2024-09-21T15:34:32.785278+0000","last_fullsized":"2024-09-21T15:34:32.785278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":17839064,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":9558,"num_read_kb":9255,"num_write":307,"num_write_kb":5274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"100'174","reported_seq":2297,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786806+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:32.786806+0000","last_peered":"2024-09-21T15:34:32.786806+0000","last_clean":"2024-09-21T15:34:32.786806+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:32.786806+0000","last_undegraded":"2024-09-21T15:34:32.786806+0000","last_fullsized":"2024-09-21T15:34:32.786806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":18286207,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":2123,"num_read_kb":11395,"num_write":173,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"100'551","reported_seq":33234,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.739552+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:32.739552+0000","last_peered":"2024-09-21T15:34:32.739552+0000","last_clean":"2024-09-21T15:34:32.739552+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:32.739552+0000","last_undegraded":"2024-09-21T15:34:32.739552+0000","last_fullsized":"2024-09-21T15:34:32.739552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":63587805,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":34585,"num_read_kb":19435,"num_write":788,"num_write_kb":4959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"100'391","reported_seq":911,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.713116+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:32.713116+0000","last_peered":"2024-09-21T15:34:32.713116+0000","last_clean":"2024-09-21T15:34:32.713116+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:32.713116+0000","last_undegraded":"2024-09-21T15:34:32.713116+0000","last_fullsized":"2024-09-21T15:34:32.713116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":17567475,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":2042,"num_read_kb":4674,"num_write":552,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"100'578","reported_seq":1779,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.747714+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:32.747714+0000","last_peered":"2024-09-21T15:34:32.747714+0000","last_clean":"2024-09-21T15:34:32.747714+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:32.747714+0000","last_undegraded":"2024-09-21T15:34:32.747714+0000","last_fullsized":"2024-09-21T15:34:32.747714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":29991782,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3846,"num_read_kb":9634,"num_write":888,"num_write_kb":7466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'291","reported_seq":5759,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.828982+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:31.828982+0000","last_peered":"2024-09-21T15:34:31.828982+0000","last_clean":"2024-09-21T15:34:31.828982+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:31.828982+0000","last_undegraded":"2024-09-21T15:34:31.828982+0000","last_fullsized":"2024-09-21T15:34:31.828982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"100'232","reported_seq":4953,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-21T15:34:32.775097+0000","last_change":"2024-09-21T15:34:33.682238+0000","last_active":"2024-09-21T15:34:33.682238+0000","last_peered":"2024-09-21T15:34:32.775097+0000","last_clean":"2024-09-21T15:34:24.521559+0000","last_became_active":"2024-09-21T15:34:31.835981+0000","last_became_peered":"2024-09-21T15:34:31.835981+0000","last_unstale":"2024-09-21T15:34:32.775097+0000","last_undegraded":"2024-09-21T15:34:32.775097+0000","last_fullsized":"2024-09-21T15:34:31.779671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":11602304,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":4812,"num_read_kb":8549,"num_write":269,"num_write_kb":10345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"101'524","reported_seq":1657,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786192+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:32.786192+0000","last_peered":"2024-09-21T15:34:32.786192+0000","last_clean":"2024-09-21T15:34:32.786192+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:32.786192+0000","last_undegraded":"2024-09-21T15:34:32.786192+0000","last_fullsized":"2024-09-21T15:34:32.786192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":28098524,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":3555,"num_read_kb":6006,"num_write":671,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.780356+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:34:31.780356+0000","last_peered":"2024-09-21T15:34:31.780356+0000","last_clean":"2024-09-21T15:34:31.780356+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:34:31.780356+0000","last_undegraded":"2024-09-21T15:34:31.780356+0000","last_fullsized":"2024-09-21T15:34:31.780356+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'299","reported_seq":3527,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.632594+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:32.632594+0000","last_peered":"2024-09-21T15:34:32.632594+0000","last_clean":"2024-09-21T15:34:32.632594+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:32.632594+0000","last_undegraded":"2024-09-21T15:34:32.632594+0000","last_fullsized":"2024-09-21T15:34:32.632594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":5364,"num_read_kb":10540,"num_write":279,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.783586+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:31.783586+0000","last_peered":"2024-09-21T15:34:31.783586+0000","last_clean":"2024-09-21T15:34:31.783586+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:31.783586+0000","last_undegraded":"2024-09-21T15:34:31.783586+0000","last_fullsized":"2024-09-21T15:34:31.783586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"101'181","reported_seq":1198,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784239+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:32.784239+0000","last_peered":"2024-09-21T15:34:32.784239+0000","last_clean":"2024-09-21T15:34:32.784239+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:32.784239+0000","last_undegraded":"2024-09-21T15:34:32.784239+0000","last_fullsized":"2024-09-21T15:34:32.784239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":37229564,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":1378,"num_read_kb":21000,"num_write":201,"num_write_kb":18044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.834198+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:31.834198+0000","last_peered":"2024-09-21T15:34:31.834198+0000","last_clean":"2024-09-21T15:34:31.834198+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:31.834198+0000","last_undegraded":"2024-09-21T15:34:31.834198+0000","last_fullsized":"2024-09-21T15:34:31.834198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"101'169","reported_seq":719,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.788333+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:32.788333+0000","last_peered":"2024-09-21T15:34:32.788333+0000","last_clean":"2024-09-21T15:34:32.788333+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:32.788333+0000","last_undegraded":"2024-09-21T15:34:32.788333+0000","last_fullsized":"2024-09-21T15:34:32.788333+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":16556525,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":806,"num_read_kb":4007,"num_write":174,"num_write_kb":2721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.783445+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:31.783445+0000","last_peered":"2024-09-21T15:34:31.783445+0000","last_clean":"2024-09-21T15:34:31.783445+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:31.783445+0000","last_undegraded":"2024-09-21T15:34:31.783445+0000","last_fullsized":"2024-09-21T15:34:31.783445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'376","reported_seq":769,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829545+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:31.829545+0000","last_peered":"2024-09-21T15:34:31.829545+0000","last_clean":"2024-09-21T15:34:31.829545+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:31.829545+0000","last_undegraded":"2024-09-21T15:34:31.829545+0000","last_fullsized":"2024-09-21T15:34:31.829545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":12612201,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":1449,"num_read_kb":4766,"num_write":486,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"101'187","reported_seq":5997,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.786988+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:32.786988+0000","last_peered":"2024-09-21T15:34:32.786988+0000","last_clean":"2024-09-21T15:34:32.786988+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:32.786988+0000","last_undegraded":"2024-09-21T15:34:32.786988+0000","last_fullsized":"2024-09-21T15:34:32.786988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":23544973,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":5793,"num_read_kb":11966,"num_write":212,"num_write_kb":4008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":98,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829722+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:31.829722+0000","last_peered":"2024-09-21T15:34:31.829722+0000","last_clean":"2024-09-21T15:34:31.829722+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:31.829722+0000","last_undegraded":"2024-09-21T15:34:31.829722+0000","last_fullsized":"2024-09-21T15:34:31.829722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'669","reported_seq":1799,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.808154+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:32.808154+0000","last_peered":"2024-09-21T15:34:32.808154+0000","last_clean":"2024-09-21T15:34:32.808154+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:32.808154+0000","last_undegraded":"2024-09-21T15:34:32.808154+0000","last_fullsized":"2024-09-21T15:34:32.808154+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":25820728,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":4347,"num_read_kb":5726,"num_write":952,"num_write_kb":10623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.781274+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:31.781274+0000","last_peered":"2024-09-21T15:34:31.781274+0000","last_clean":"2024-09-21T15:34:31.781274+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:31.781274+0000","last_undegraded":"2024-09-21T15:34:31.781274+0000","last_fullsized":"2024-09-21T15:34:31.781274+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'426","reported_seq":2461,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.801306+0000","last_change":"2024-09-21T15:34:29.764833+0000","last_active":"2024-09-21T15:34:32.801306+0000","last_peered":"2024-09-21T15:34:32.801306+0000","last_clean":"2024-09-21T15:34:32.801306+0000","last_became_active":"2024-09-21T15:34:29.762828+0000","last_became_peered":"2024-09-21T15:34:29.762828+0000","last_unstale":"2024-09-21T15:34:32.801306+0000","last_undegraded":"2024-09-21T15:34:32.801306+0000","last_fullsized":"2024-09-21T15:34:32.801306+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:17.503095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":24984429,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":3335,"num_read_kb":8883,"num_write":625,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.831473+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:31.831473+0000","last_peered":"2024-09-21T15:34:31.831473+0000","last_clean":"2024-09-21T15:34:31.831473+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:31.831473+0000","last_undegraded":"2024-09-21T15:34:31.831473+0000","last_fullsized":"2024-09-21T15:34:31.831473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'203","reported_seq":705,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.966656+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:31.966656+0000","last_peered":"2024-09-21T15:34:31.966656+0000","last_clean":"2024-09-21T15:34:31.966656+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:31.966656+0000","last_undegraded":"2024-09-21T15:34:31.966656+0000","last_fullsized":"2024-09-21T15:34:31.966656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":23697932,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":991,"num_read_kb":3966,"num_write":190,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":121,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.829457+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:31.829457+0000","last_peered":"2024-09-21T15:34:31.829457+0000","last_clean":"2024-09-21T15:34:31.829457+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:31.829457+0000","last_undegraded":"2024-09-21T15:34:31.829457+0000","last_fullsized":"2024-09-21T15:34:31.829457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":101,"state":"peering","last_fresh":"2024-09-21T15:34:31.845196+0000","last_change":"2024-09-21T15:34:33.682082+0000","last_active":"2024-09-21T15:34:33.682082+0000","last_peered":"2024-09-21T15:34:31.845196+0000","last_clean":"2024-09-21T15:34:28.744238+0000","last_became_active":"2024-09-21T15:34:31.832616+0000","last_became_peered":"2024-09-21T15:34:31.832616+0000","last_unstale":"2024-09-21T15:34:31.845196+0000","last_undegraded":"2024-09-21T15:34:31.845196+0000","last_fullsized":"2024-09-21T15:34:31.777222+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:57.645467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"100'474","reported_seq":1391,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.688193+0000","last_change":"2024-09-21T15:34:29.765868+0000","last_active":"2024-09-21T15:34:32.688193+0000","last_peered":"2024-09-21T15:34:32.688193+0000","last_clean":"2024-09-21T15:34:32.688193+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:34:32.688193+0000","last_undegraded":"2024-09-21T15:34:32.688193+0000","last_fullsized":"2024-09-21T15:34:32.688193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":12931068,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2587,"num_read_kb":6802,"num_write":662,"num_write_kb":5716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'454","reported_seq":1505,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.663535+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:32.663535+0000","last_peered":"2024-09-21T15:34:32.663535+0000","last_clean":"2024-09-21T15:34:32.663535+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:32.663535+0000","last_undegraded":"2024-09-21T15:34:32.663535+0000","last_fullsized":"2024-09-21T15:34:32.663535+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522187,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":3121,"num_read_kb":7563,"num_write":615,"num_write_kb":3711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"101'263","reported_seq":4752,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.787511+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:32.787511+0000","last_peered":"2024-09-21T15:34:32.787511+0000","last_clean":"2024-09-21T15:34:32.787511+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:32.787511+0000","last_undegraded":"2024-09-21T15:34:32.787511+0000","last_fullsized":"2024-09-21T15:34:32.787511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":21750935,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4749,"num_read_kb":13714,"num_write":287,"num_write_kb":5517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"100'297","reported_seq":4678,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.636499+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:32.636499+0000","last_peered":"2024-09-21T15:34:32.636499+0000","last_clean":"2024-09-21T15:34:32.636499+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:32.636499+0000","last_undegraded":"2024-09-21T15:34:32.636499+0000","last_fullsized":"2024-09-21T15:34:32.636499+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":6030,"num_read_kb":11328,"num_write":324,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"101'221","reported_seq":1956,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.790627+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:32.790627+0000","last_peered":"2024-09-21T15:34:32.790627+0000","last_clean":"2024-09-21T15:34:32.790627+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:32.790627+0000","last_undegraded":"2024-09-21T15:34:32.790627+0000","last_fullsized":"2024-09-21T15:34:32.790627+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":35350897,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":1818,"num_read_kb":8299,"num_write":229,"num_write_kb":21213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17295,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.834220+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:31.834220+0000","last_peered":"2024-09-21T15:34:31.834220+0000","last_clean":"2024-09-21T15:34:31.834220+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:31.834220+0000","last_undegraded":"2024-09-21T15:34:31.834220+0000","last_fullsized":"2024-09-21T15:34:31.834220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17967,"num_read_kb":4981,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"101'283","reported_seq":33608,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.784034+0000","last_change":"2024-09-21T15:34:30.767437+0000","last_active":"2024-09-21T15:34:32.784034+0000","last_peered":"2024-09-21T15:34:32.784034+0000","last_clean":"2024-09-21T15:34:32.784034+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:34:32.784034+0000","last_undegraded":"2024-09-21T15:34:32.784034+0000","last_fullsized":"2024-09-21T15:34:32.784034+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":33828,"num_read_kb":32899,"num_write":318,"num_write_kb":20815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"100'303","reported_seq":1574,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-21T15:34:32.685973+0000","last_change":"2024-09-21T15:34:33.682386+0000","last_active":"2024-09-21T15:34:33.682386+0000","last_peered":"2024-09-21T15:34:32.685973+0000","last_clean":"2024-09-21T15:34:25.286842+0000","last_became_active":"2024-09-21T15:34:31.838100+0000","last_became_peered":"2024-09-21T15:34:31.838100+0000","last_unstale":"2024-09-21T15:34:32.685973+0000","last_undegraded":"2024-09-21T15:34:32.685973+0000","last_fullsized":"2024-09-21T15:34:32.685973+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":30043907,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":2337,"num_read_kb":6342,"num_write":346,"num_write_kb":3654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":732,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:31.831304+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:31.831304+0000","last_peered":"2024-09-21T15:34:31.831304+0000","last_clean":"2024-09-21T15:34:31.831304+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:31.831304+0000","last_undegraded":"2024-09-21T15:34:31.831304+0000","last_fullsized":"2024-09-21T15:34:31.831304+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"100'397","reported_seq":5729,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.087367+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:32.087367+0000","last_peered":"2024-09-21T15:34:32.087367+0000","last_clean":"2024-09-21T15:34:32.087367+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:32.087367+0000","last_undegraded":"2024-09-21T15:34:32.087367+0000","last_fullsized":"2024-09-21T15:34:32.087367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":26255034,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":11,"num_read":7440,"num_read_kb":18349,"num_write":533,"num_write_kb":18015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"100'459","reported_seq":7114,"reported_epoch":101,"state":"peering","last_fresh":"2024-09-21T15:34:32.785184+0000","last_change":"2024-09-21T15:34:32.785387+0000","last_active":"2024-09-21T15:34:32.785184+0000","last_peered":"2024-09-21T15:34:32.607757+0000","last_clean":"2024-09-21T15:34:26.155415+0000","last_became_active":"2024-09-21T15:34:31.842645+0000","last_became_peered":"2024-09-21T15:34:31.842645+0000","last_unstale":"2024-09-21T15:34:32.785184+0000","last_undegraded":"2024-09-21T15:34:32.785184+0000","last_fullsized":"2024-09-21T15:34:32.785184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:15.523620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":20247120,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":7026,"num_read_kb":10500,"num_write":608,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2138377,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"100'312","reported_seq":1818,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-21T15:34:32.774298+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:32.774298+0000","last_peered":"2024-09-21T15:34:32.774298+0000","last_clean":"2024-09-21T15:34:32.774298+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:32.774298+0000","last_undegraded":"2024-09-21T15:34:32.774298+0000","last_fullsized":"2024-09-21T15:34:32.774298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":29086991,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":4099,"num_read_kb":5101,"num_write":301,"num_write_kb":10372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":857333592,"num_objects":1340,"num_object_clones":296,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":469,"num_read":197147,"num_read_kb":316647,"num_write":13327,"num_write_kb":294048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":19186363,"num_keys_recovered":127,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184414208,"data_stored":220814343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10567,"ondisk_log_size":10567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63848,"kb_used_data":32732,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94307992,"statfs":{"total":96636764160,"available":96571383808,"internally_reserved":0,"allocated":33517568,"data_stored":37973447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.59799999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.77600000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.86899999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.98399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":1.1359999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.181},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.095}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":12640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":12943360,"data_stored":14264340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.437},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.501}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.68200000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.82799999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.98199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.871},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":1.0589999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.036},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":1.0089999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":24016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":24592384,"data_stored":28961867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":1.0269999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.1379999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.226},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.194}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.26},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1679999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.411},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.29}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67220,"kb_used_data":36100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94304620,"statfs":{"total":96636764160,"available":96567930880,"internally_reserved":0,"allocated":36966400,"data_stored":41423229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.96099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.627},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.67700000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.75800000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.72499999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.01}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.82999999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":1.046},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.79800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77420,"kb_used_data":38108,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94294420,"statfs":{"total":96636764160,"available":96557486080,"internally_reserved":0,"allocated":39022592,"data_stored":48012790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.624},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.499}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.85399999999999998},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":0.745}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.88700000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.71099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42116,"kb_used_data":6908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329724,"statfs":{"total":96636764160,"available":96593637376,"internally_reserved":0,"allocated":7073792,"data_stored":7448762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.62}]},{"osd":1,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.375},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0509999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.83999999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.90500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53784,"kb_used_data":14476,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94318056,"statfs":{"total":96636764160,"available":96581689344,"internally_reserved":0,"allocated":14823424,"data_stored":17225723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.626}]},{"osd":2,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.434},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.70799999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.745},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53268,"kb_used_data":26244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318572,"statfs":{"total":96636764160,"available":96582217728,"internally_reserved":0,"allocated":26873856,"data_stored":31740831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.41699999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83499999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.91100000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.95799999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25079808,"data_stored":30593335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13619200,"data_stored":16668576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5873664,"data_stored":6891634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37822464,"data_stored":47455662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35766272,"data_stored":40866101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23388160,"data_stored":28404720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11149312,"data_stored":13116844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":31715328,"data_stored":36817471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:36.731 INFO:teuthology.orchestra.run.smithi107.stdout:5065 punch from 0x4176b0 to 0x419e3c, (0x278c bytes) 2024-09-21T15:34:36.731 INFO:teuthology.orchestra.run.smithi107.stdout:5066 read 0x4ad680 thru 0x4b6dca (0x974b bytes) 2024-09-21T15:34:36.731 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:36.731 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:36.732 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:36.732 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:36.732 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:36.732 INFO:teuthology.orchestra.run.smithi107.stdout:5067 read 0x471d3d thru 0x474cb8 (0x2f7c bytes) 2024-09-21T15:34:36.732 INFO:teuthology.orchestra.run.smithi107.stdout:5068 read 0x35c7c0 thru 0x360c61 (0x44a2 bytes) 2024-09-21T15:34:36.734 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.734 INFO:teuthology.orchestra.run.smithi107.stdout:5070 read 0x71e2e0 thru 0x7265d2 (0x82f3 bytes) 2024-09-21T15:34:36.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:36.748 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:36.746+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:36.769 INFO:teuthology.orchestra.run.smithi107.stdout:5072 write 0xaf8f58b thru 0xaf9a998 (0xb40e bytes) 2024-09-21T15:34:36.788 INFO:teuthology.orchestra.run.smithi107.stdout:5074 writesame 0xd26d15c thru 0xd26ebee data_size 0x1(0x1a93 bytes) 2024-09-21T15:34:36.804 INFO:teuthology.orchestra.run.smithi107.stdout:5075 trunc from 0xd26ebef to 0x71bc93a 2024-09-21T15:34:36.827 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:34:36.828 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.828 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.828 INFO:teuthology.orchestra.run.smithi107.stdout:5079 punch from 0x5827df3 to 0x582afaf, (0x31bc bytes) 2024-09-21T15:34:36.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:36.847 INFO:teuthology.orchestra.run.smithi107.stdout:5080 writesame 0xaaa8f57 thru 0xaaa99e2 data_size 0x2d(0xa8c bytes) 2024-09-21T15:34:36.847 INFO:teuthology.orchestra.run.smithi107.stdout:5081 write 0x5eb3999 thru 0x5eb7d9b (0x4403 bytes) 2024-09-21T15:34:36.848 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:36.846+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:36.852 INFO:teuthology.orchestra.run.smithi107.stdout:5082 read 0x7d8a838 thru 0x7d9a137 (0xf900 bytes) 2024-09-21T15:34:36.867 INFO:teuthology.orchestra.run.smithi107.stdout:5084 writesame 0xdb995a0 thru 0xdba670b data_size 0x1a(0xd16c bytes) 2024-09-21T15:34:36.867 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.867 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.867 INFO:teuthology.orchestra.run.smithi107.stdout:5087 write 0x8b8879b thru 0x8b8dc88 (0x54ee bytes) 2024-09-21T15:34:36.872 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.873 INFO:teuthology.orchestra.run.smithi107.stdout:5089 trunc from 0xdba670c to 0xe539079 2024-09-21T15:34:36.881 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.220576864000002, 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:34:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:34:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:34:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:34:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.216726282, 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:34:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:34:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:34:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:34:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.216701090000001, 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:34:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:34:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.212725068000001, 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:34:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.212699046000001, 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:34:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.212675492999999, 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.212640651000001, 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.208659129000001, 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.208551752000002, 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.208525182999999, 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.208497607000002, 2024-09-21T15:34:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:34:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.208474097, 2024-09-21T15:34:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204772470999998, 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:34:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:34:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:34:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204742717999999, 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204715649000001, 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:34:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204692312999999, 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:34:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204623709, 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.204594968999999, 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.200681985999999, 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.954 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:34:36.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.957 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 28.200626914000001, 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:36.961 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:36.963 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5090 punch from 0x18ae247 to 0x18b023a, (0x1ff3 bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5093 read 0x31a2e33 thru 0x31b16a0 (0xe86e bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5094 read 0x9100f0 thru 0x910710 (0x621 bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5095 read 0x8d8e5ad thru 0x8d9d975 (0xf3c9 bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5096 read 0x7850f0c thru 0x7859377 (0x846c bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5099 read 0x8b645a2 thru 0x8b69dcc (0x582b bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5100 trunc from 0xe539079 to 0xb7ffe27 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5101 read 0x5f7f55f thru 0x5f8d7f1 (0xe293 bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5102 trunc from 0xb7ffe27 to 0x4d0414 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5104 read 0x152c4c thru 0x15b2f8 (0x86ad bytes) 2024-09-21T15:34:36.964 INFO:teuthology.orchestra.run.smithi107.stdout:5105 write 0xea62750 thru 0xea634ae (0xd5f bytes) 2024-09-21T15:34:36.965 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:36.958+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:36.975 INFO:teuthology.orchestra.run.smithi107.stdout:5106 trunc from 0xea634af to 0x738cf18 2024-09-21T15:34:36.985 INFO:teuthology.orchestra.run.smithi107.stdout:5107 punch from 0x1c0aa27 to 0x1c0ba75, (0x104e bytes) 2024-09-21T15:34:36.992 INFO:teuthology.orchestra.run.smithi107.stdout:5108 trunc from 0x738cf18 to 0xbc9136e 2024-09-21T15:34:36.993 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:36.999 INFO:teuthology.orchestra.run.smithi107.stdout:5109 writesame 0x9527f86 thru 0x95310f0 data_size 0x1(0x916b bytes) 2024-09-21T15:34:37.010 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:37.010 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:37.026 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:37.027 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:37.032 INFO:teuthology.orchestra.run.smithi107.stdout:5112 trunc from 0xbc9136e to 0xd261617 2024-09-21T15:34:37.039 INFO:teuthology.orchestra.run.smithi107.stdout:5113 punch from 0x60ec657 to 0x60f10e3, (0x4a8c bytes) 2024-09-21T15:34:37.039 INFO:teuthology.orchestra.run.smithi107.stdout:5114 write 0xb9b0280 thru 0xb9bb1f7 (0xaf78 bytes) 2024-09-21T15:34:37.044 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:37.054 INFO:teuthology.orchestra.run.smithi107.stdout:5116 trunc from 0xd261617 to 0x6b70285 2024-09-21T15:34:37.063 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:37.058+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:37.071 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:37.072 INFO:teuthology.orchestra.run.smithi107.stdout:5117 write 0x27974fd thru 0x2797a6d (0x571 bytes) 2024-09-21T15:34:37.077 INFO:teuthology.orchestra.run.smithi107.stdout:5119 read 0x9627ac thru 0x9680cb (0x5920 bytes) 2024-09-21T15:34:37.077 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.078 INFO:teuthology.orchestra.run.smithi107.stdout:5122 read 0x12af999 thru 0x12bdd55 (0xe3bd bytes) 2024-09-21T15:34:37.092 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:37.098 INFO:teuthology.orchestra.run.smithi107.stdout:5123 write 0xd027aa7 thru 0xd027ba5 (0xff bytes) 2024-09-21T15:34:37.098 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:37.098 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:37.098 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:37.098 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:37.102 INFO:teuthology.orchestra.run.smithi107.stdout:5124 writesame 0x6bb439 thru 0x6cac10 data_size 0x38(0xf7d8 bytes) 2024-09-21T15:34:37.106 INFO:teuthology.orchestra.run.smithi107.stdout:5125 read 0x24bf3d9 thru 0x24cb848 (0xc470 bytes) 2024-09-21T15:34:37.106 INFO:teuthology.orchestra.run.smithi107.stdout:5127 read 0x2307029 thru 0x2309825 (0x27fd bytes) 2024-09-21T15:34:37.107 INFO:teuthology.orchestra.run.smithi107.stdout:5128 write 0x3bc6595 thru 0x3bc95cb (0x3037 bytes) 2024-09-21T15:34:37.112 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:37.114 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:37.115 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:37.124 INFO:teuthology.orchestra.run.smithi107.stdout:5130 trunc from 0xd027ba6 to 0x515d502 2024-09-21T15:34:37.135 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:37.135 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:37.158 INFO:teuthology.orchestra.run.smithi107.stdout:5132 write 0xc3f5788 thru 0xc40318e (0xda07 bytes) 2024-09-21T15:34:37.163 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:37.160+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:37.164 INFO:teuthology.orchestra.run.smithi107.stdout:5134 write 0x1d76ae2 thru 0x1d7734f (0x86e bytes) 2024-09-21T15:34:37.168 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.168 INFO:teuthology.orchestra.run.smithi107.stdout:5137 write 0x6b71a4b thru 0x6b80819 (0xedcf bytes) 2024-09-21T15:34:37.172 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.172 INFO:teuthology.orchestra.run.smithi107.stdout:5139 write 0x5612b4f thru 0x5621e13 (0xf2c5 bytes) 2024-09-21T15:34:37.176 INFO:teuthology.orchestra.run.smithi107.stdout:5140 write 0x40d3bb6 thru 0x40e0520 (0xc96b bytes) 2024-09-21T15:34:37.180 INFO:teuthology.orchestra.run.smithi107.stdout:5141 read 0x5872628 thru 0x5874035 (0x1a0e bytes) 2024-09-21T15:34:37.195 INFO:teuthology.orchestra.run.smithi107.stdout:5142 trunc from 0xc40318f to 0x2fed40e 2024-09-21T15:34:37.214 INFO:teuthology.orchestra.run.smithi107.stdout:5143 read 0x263702a thru 0x2644fcb (0xdfa2 bytes) 2024-09-21T15:34:37.233 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:37.233 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:37.246 INFO:teuthology.orchestra.run.smithi107.stdout:5145 write 0xbc65752 thru 0xbc709f6 (0xb2a5 bytes) 2024-09-21T15:34:37.248 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:37.253 INFO:teuthology.orchestra.run.smithi107.stdout:5146 trunc from 0xbc709f7 to 0xae04f0a 2024-09-21T15:34:37.266 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:37.262+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:37.287 INFO:teuthology.orchestra.run.smithi107.stdout:5147 writesame 0xbaf645c thru 0xbafca17 data_size 0x22(0x65bc bytes) 2024-09-21T15:34:37.287 INFO:teuthology.orchestra.run.smithi107.stdout:5148 writesame 0x24204ba thru 0x242395f data_size 0x17(0x34a6 bytes) 2024-09-21T15:34:37.290 INFO:teuthology.orchestra.run.smithi107.stdout:5149 punch from 0x308f5ce to 0x3092bb1, (0x35e3 bytes) 2024-09-21T15:34:37.290 INFO:teuthology.orchestra.run.smithi107.stdout:5150 punch from 0x2cef98a to 0x2cfb8c0, (0xbf36 bytes) 2024-09-21T15:34:37.290 INFO:teuthology.orchestra.run.smithi107.stdout:5151 punch from 0x50623e3 to 0x506d813, (0xb430 bytes) 2024-09-21T15:34:37.306 INFO:teuthology.orchestra.run.smithi107.stdout:5152 writesame 0xd5d4b28 thru 0xd5e0f22 data_size 0x1(0xc3fb bytes) 2024-09-21T15:34:37.306 INFO:teuthology.orchestra.run.smithi107.stdout:5153 write 0x4012ec9 thru 0x4021d4a (0xee82 bytes) 2024-09-21T15:34:37.312 INFO:teuthology.orchestra.run.smithi107.stdout:5154 read 0x16c5aa9 thru 0x16d4dca (0xf322 bytes) 2024-09-21T15:34:37.313 INFO:teuthology.orchestra.run.smithi107.stdout:5156 punch from 0x91ff97e to 0x92067c9, (0x6e4b bytes) 2024-09-21T15:34:37.313 INFO:teuthology.orchestra.run.smithi107.stdout:5157 read 0x4e397c5 thru 0x4e3ba9f (0x22db bytes) 2024-09-21T15:34:37.314 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.314 INFO:teuthology.orchestra.run.smithi107.stdout:5161 read 0x20ba44c thru 0x20c381b (0x93d0 bytes) 2024-09-21T15:34:37.314 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.314 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.315 INFO:teuthology.orchestra.run.smithi107.stdout:5164 write 0x81ba9eb thru 0x81c0142 (0x5758 bytes) 2024-09-21T15:34:37.320 INFO:teuthology.orchestra.run.smithi107.stdout:5165 writesame 0x24e1538 thru 0x24ec18a data_size 0x33(0xac53 bytes) 2024-09-21T15:34:37.323 INFO:teuthology.orchestra.run.smithi107.stdout:5166 punch from 0x1bd4ab0 to 0x1bdc897, (0x7de7 bytes) 2024-09-21T15:34:37.323 INFO:teuthology.orchestra.run.smithi107.stdout:5167 read 0x1e7788 thru 0x1e9411 (0x1c8a bytes) 2024-09-21T15:34:37.323 INFO:teuthology.orchestra.run.smithi107.stdout:5169 write 0x2896553 thru 0x28a0fce (0xaa7c bytes) 2024-09-21T15:34:37.326 INFO:teuthology.orchestra.run.smithi107.stdout:5171 write 0x7619343 thru 0x761f15c (0x5e1a bytes) 2024-09-21T15:34:37.331 INFO:teuthology.orchestra.run.smithi107.stdout:5172 write 0x10f2d58 thru 0x10f971f (0x69c8 bytes) 2024-09-21T15:34:37.333 INFO:teuthology.orchestra.run.smithi107.stdout:5173 writesame 0x32ac4ba thru 0x32b04c0 data_size 0x1(0x4007 bytes) 2024-09-21T15:34:37.348 INFO:teuthology.orchestra.run.smithi107.stdout:5175 read 0xacdb686 thru 0xacdbd80 (0x6fb bytes) 2024-09-21T15:34:37.349 INFO:teuthology.orchestra.run.smithi107.stdout:5176 read 0x6d45496 thru 0x6d474a5 (0x2010 bytes) 2024-09-21T15:34:37.349 INFO:teuthology.orchestra.run.smithi107.stdout:5177 punch from 0x2e9845c to 0x2ea578b, (0xd32f bytes) 2024-09-21T15:34:37.366 INFO:teuthology.orchestra.run.smithi107.stdout:5178 trunc from 0xd5e0f23 to 0x6c37dec 2024-09-21T15:34:37.367 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:37.364+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:37.387 INFO:teuthology.orchestra.run.smithi107.stdout:5179 read 0x6b9c1f6 thru 0x6ba1367 (0x5172 bytes) 2024-09-21T15:34:37.412 INFO:teuthology.orchestra.run.smithi107.stdout:5181 write 0xb13cd60 thru 0xb14afcc (0xe26d bytes) 2024-09-21T15:34:37.419 INFO:teuthology.orchestra.run.smithi107.stdout:5182 read 0xbe9a5 thru 0xc30ae (0x470a bytes) 2024-09-21T15:34:37.421 INFO:teuthology.orchestra.run.smithi107.stdout:5183 read 0x864d824 thru 0x865c27b (0xea58 bytes) 2024-09-21T15:34:37.421 INFO:teuthology.orchestra.run.smithi107.stdout:5184 read 0xb76c2c thru 0xb77e4c (0x1221 bytes) 2024-09-21T15:34:37.422 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:37.422 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:37.424 INFO:teuthology.orchestra.run.smithi107.stdout:5185 trunc from 0xb14afcd to 0xa6484fd 2024-09-21T15:34:37.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:37.447 INFO:teuthology.orchestra.run.smithi107.stdout:5186 trunc from 0xa6484fd to 0xaa4afa7 2024-09-21T15:34:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:5188 write 0x1effc0e thru 0x1f0b75a (0xbb4d bytes) 2024-09-21T15:34:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:5189 read 0x3c63b6a thru 0x3c65e66 (0x22fd bytes) 2024-09-21T15:34:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:5191 write 0x379dc20 thru 0x379e61c (0x9fd bytes) 2024-09-21T15:34:37.460 INFO:teuthology.orchestra.run.smithi107.stdout:5193 read 0x31ad08c thru 0x31b2166 (0x50db bytes) 2024-09-21T15:34:37.460 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.468 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:37.466+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:37.470 INFO:teuthology.orchestra.run.smithi107.stdout:5196 write 0xb6857f1 thru 0xb68a600 (0x4e10 bytes) 2024-09-21T15:34:37.473 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.474 INFO:teuthology.orchestra.run.smithi107.stdout:5200 write 0x8674e66 thru 0x8679805 (0x49a0 bytes) 2024-09-21T15:34:37.478 INFO:teuthology.orchestra.run.smithi107.stdout:5201 write 0x7df7b36 thru 0x7dfc45f (0x492a bytes) 2024-09-21T15:34:37.491 INFO:teuthology.orchestra.run.smithi107.stdout:5203 trunc from 0xb68a601 to 0x700dc5f 2024-09-21T15:34:37.520 INFO:teuthology.orchestra.run.smithi107.stdout:5204 write 0x9cd10aa thru 0x9cd3a5f (0x29b6 bytes) 2024-09-21T15:34:37.544 INFO:teuthology.orchestra.run.smithi107.stdout:5205 writesame 0xeb8916e thru 0xeb8e7d2 data_size 0x1(0x5665 bytes) 2024-09-21T15:34:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:5210 punch from 0x81d0a3d to 0x81d5400, (0x49c3 bytes) 2024-09-21T15:34:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:5211 write 0xc5947b4 thru 0xc59f587 (0xadd4 bytes) 2024-09-21T15:34:37.554 INFO:teuthology.orchestra.run.smithi107.stdout:5212 write 0x68ae207 thru 0x68bcfb9 (0xedb3 bytes) 2024-09-21T15:34:37.569 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:37.566+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:37.580 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:37.580 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:37.586 INFO:teuthology.orchestra.run.smithi107.stdout:5213 trunc from 0xeb8e7d3 to 0x1835e1a 2024-09-21T15:34:37.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:37.618 INFO:teuthology.orchestra.run.smithi107.stdout:5214 write 0xf178f thru 0xf38c8 (0x213a bytes) 2024-09-21T15:34:37.621 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.644 INFO:teuthology.orchestra.run.smithi107.stdout:5216 trunc from 0x1835e1a to 0xdb346c8 2024-09-21T15:34:37.655 INFO:teuthology.orchestra.run.smithi107.stdout:5217 write 0x3d9aad8 thru 0x3d9d58e (0x2ab7 bytes) 2024-09-21T15:34:37.662 INFO:teuthology.orchestra.run.smithi107.stdout:5218 read 0xca8c272 thru 0xca95990 (0x971f bytes) 2024-09-21T15:34:37.662 INFO:teuthology.orchestra.run.smithi107.stdout:5219 read 0xb4346b0 thru 0xb43852e (0x3e7f bytes) 2024-09-21T15:34:37.664 INFO:teuthology.orchestra.run.smithi107.stdout:5220 trunc from 0xdb346c8 to 0xcf50244 2024-09-21T15:34:37.670 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:37.668+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:37.675 INFO:teuthology.orchestra.run.smithi107.stdout:5221 write 0x25d44b0 thru 0x25dadbf (0x6910 bytes) 2024-09-21T15:34:37.682 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.682 INFO:teuthology.orchestra.run.smithi107.stdout:5223 read 0x1e8b320 thru 0x1e94e31 (0x9b12 bytes) 2024-09-21T15:34:37.682 INFO:teuthology.orchestra.run.smithi107.stdout:5224 write 0xbce76a0 thru 0xbcf6ae5 (0xf446 bytes) 2024-09-21T15:34:37.689 INFO:teuthology.orchestra.run.smithi107.stdout:5227 punch from 0xfa652a to 0xfa96d0, (0x31a6 bytes) 2024-09-21T15:34:37.690 INFO:teuthology.orchestra.run.smithi107.stdout:5229 writesame 0xaa7b012 thru 0xaa7ee14 data_size 0x19(0x3e03 bytes) 2024-09-21T15:34:37.697 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.697 INFO:teuthology.orchestra.run.smithi107.stdout:5231 write 0x86ff29a thru 0x8708a60 (0x97c7 bytes) 2024-09-21T15:34:37.704 INFO:teuthology.orchestra.run.smithi107.stdout:5233 punch from 0x1957d to 0x21dee, (0x8871 bytes) 2024-09-21T15:34:37.704 INFO:teuthology.orchestra.run.smithi107.stdout:5234 write 0x89e4699 thru 0x89f35f2 (0xef5a bytes) 2024-09-21T15:34:37.707 INFO:teuthology.orchestra.run.smithi107.stdout:5235 read 0xb89ac1e thru 0xb8a14e8 (0x68cb bytes) 2024-09-21T15:34:37.709 INFO:teuthology.orchestra.run.smithi107.stdout:5237 trunc from 0xcf50244 to 0xd728739 2024-09-21T15:34:37.721 INFO:teuthology.orchestra.run.smithi107.stdout:5238 punch from 0x88349f9 to 0x883c104, (0x770b bytes) 2024-09-21T15:34:37.736 INFO:teuthology.orchestra.run.smithi107.stdout:5239 trunc from 0xd728739 to 0x6ac9fbe 2024-09-21T15:34:37.739 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:37.740 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:37.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:37.765 INFO:teuthology.orchestra.run.smithi107.stdout:5240 read 0x4d0a0d1 thru 0x4d1211c (0x804c bytes) 2024-09-21T15:34:37.765 INFO:teuthology.orchestra.run.smithi107.stdout:5242 writesame 0x2e652d1 thru 0x2e6ea5e data_size 0x13(0x978e bytes) 2024-09-21T15:34:37.769 INFO:teuthology.orchestra.run.smithi107.stdout:5245 writesame 0x3f8f3e thru 0x3ff2be data_size 0x7(0x6381 bytes) 2024-09-21T15:34:37.771 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:37.766+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:37.776 INFO:teuthology.orchestra.run.smithi107.stdout:5247 write 0x62087e4 thru 0x6217bf1 (0xf40e bytes) 2024-09-21T15:34:37.781 INFO:teuthology.orchestra.run.smithi107.stdout:5248 punch from 0x32836f3 to 0x3285c4f, (0x255c bytes) 2024-09-21T15:34:37.782 INFO:teuthology.orchestra.run.smithi107.stdout:5249 punch from 0x4732b8e to 0x4741629, (0xea9b bytes) 2024-09-21T15:34:37.807 INFO:teuthology.orchestra.run.smithi107.stdout:5250 writesame 0xddd69c6 thru 0xddd9706 data_size 0x7(0x2d41 bytes) 2024-09-21T15:34:37.807 INFO:teuthology.orchestra.run.smithi107.stdout:5251 write 0xc0d9eb thru 0xc1a3a6 (0xc9bc bytes) 2024-09-21T15:34:37.809 INFO:teuthology.orchestra.run.smithi107.stdout:5252 read 0x26b3720 thru 0x26bfa1a (0xc2fb bytes) 2024-09-21T15:34:37.810 INFO:teuthology.orchestra.run.smithi107.stdout:5253 read 0x1704c22 thru 0x170dfda (0x93b9 bytes) 2024-09-21T15:34:37.811 INFO:teuthology.orchestra.run.smithi107.stdout:5254 writesame 0x1b6df51 thru 0x1b7d7ce data_size 0x11(0xf87e bytes) 2024-09-21T15:34:37.817 INFO:teuthology.orchestra.run.smithi107.stdout:5255 read 0x50359 thru 0x5b0f8 (0xada0 bytes) 2024-09-21T15:34:37.818 INFO:teuthology.orchestra.run.smithi107.stdout:5256 read 0x3fe7a8f thru 0x3fedfb7 (0x6529 bytes) 2024-09-21T15:34:37.819 INFO:teuthology.orchestra.run.smithi107.stdout:5257 write 0x2c5dafc thru 0x2c68411 (0xa916 bytes) 2024-09-21T15:34:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:5258 read 0x58c8f7c thru 0x58cb6f9 (0x277e bytes) 2024-09-21T15:34:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:5260 punch from 0xcaa72f6 to 0xcaa900d, (0x1d17 bytes) 2024-09-21T15:34:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:5261 read 0x7bb9911 thru 0x7bbf173 (0x5863 bytes) 2024-09-21T15:34:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:5262 write 0x26bc2ca thru 0x26c62da (0xa011 bytes) 2024-09-21T15:34:37.827 INFO:teuthology.orchestra.run.smithi107.stdout:5263 read 0x1d41734 thru 0x1d4920a (0x7ad7 bytes) 2024-09-21T15:34:37.828 INFO:teuthology.orchestra.run.smithi107.stdout:5264 writesame 0x4d82d8e thru 0x4d8bf5b data_size 0x3(0x91ce bytes) 2024-09-21T15:34:37.839 INFO:teuthology.orchestra.run.smithi107.stdout:5265 write 0x9b8b182 thru 0x9b9451e (0x939d bytes) 2024-09-21T15:34:37.872 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:37.870+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:37.875 INFO:teuthology.orchestra.run.smithi107.stdout:5266 trunc from 0xddd9707 to 0x954142 2024-09-21T15:34:37.910 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:37.910 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:37.913 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.913 INFO:teuthology.orchestra.run.smithi107.stdout:5270 read 0x6b7458 thru 0x6bacee (0x3897 bytes) 2024-09-21T15:34:37.914 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:37.914 INFO:teuthology.orchestra.run.smithi107.stdout:5272 read 0x7539a6 thru 0x75c239 (0x8894 bytes) 2024-09-21T15:34:37.928 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:37.934 INFO:teuthology.orchestra.run.smithi107.stdout:5273 trunc from 0x954142 to 0xaf0324c 2024-09-21T15:34:37.967 INFO:teuthology.orchestra.run.smithi107.stdout:5275 trunc from 0xaf0324c to 0xc7757d 2024-09-21T15:34:37.972 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:37.968+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:5276 writesame 0xc92c173 thru 0xc93209f data_size 0xb(0x5f2d bytes) 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:5278 read 0x6de820b thru 0x6def617 (0x740d bytes) 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:5280 punch from 0x7b5f3b3 to 0x7b67302, (0x7f4f bytes) 2024-09-21T15:34:38.018 INFO:teuthology.orchestra.run.smithi107.stdout:5281 write 0xc7d644a thru 0xc7dbd1d (0x58d4 bytes) 2024-09-21T15:34:38.024 INFO:teuthology.orchestra.run.smithi107.stdout:5282 write 0x18aacc0 thru 0x18b8f0b (0xe24c bytes) 2024-09-21T15:34:38.031 INFO:teuthology.orchestra.run.smithi107.stdout:5285 write 0x8d26ad8 thru 0x8d2ac90 (0x41b9 bytes) 2024-09-21T15:34:38.038 INFO:teuthology.orchestra.run.smithi107.stdout:5286 read 0x890b777 thru 0x89124e2 (0x6d6c bytes) 2024-09-21T15:34:38.038 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.039 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.039 INFO:teuthology.orchestra.run.smithi107.stdout:5293 trunc from 0xc9320a0 to 0xc42c75f 2024-09-21T15:34:38.056 INFO:teuthology.orchestra.run.smithi107.stdout:5294 writesame 0xb97c6c thru 0xba5c34 data_size 0x1(0xdfc9 bytes) 2024-09-21T15:34:38.072 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:38.072+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:38.082 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:38.082 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:38.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:5295 write 0x121dc58 thru 0x122370b (0x5ab4 bytes) 2024-09-21T15:34:38.134 INFO:teuthology.orchestra.run.smithi107.stdout:5296 read 0xf8d3db thru 0xf8fffc (0x2c22 bytes) 2024-09-21T15:34:38.135 INFO:teuthology.orchestra.run.smithi107.stdout:5297 writesame 0x9fce5df thru 0x9fdd33a data_size 0x16(0xed5c bytes) 2024-09-21T15:34:38.144 INFO:teuthology.orchestra.run.smithi107.stdout:5298 read 0xad73ee1 thru 0xad755d6 (0x16f6 bytes) 2024-09-21T15:34:38.144 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.145 INFO:teuthology.orchestra.run.smithi107.stdout:5303 read 0x443a76c thru 0x44429f6 (0x828b bytes) 2024-09-21T15:34:38.145 INFO:teuthology.orchestra.run.smithi107.stdout:5305 write 0x84db065 thru 0x84deb14 (0x3ab0 bytes) 2024-09-21T15:34:38.147 INFO:teuthology.orchestra.run.smithi107.stdout:5307 writesame 0x3efc246 thru 0x3f01c11 data_size 0xe(0x59cc bytes) 2024-09-21T15:34:38.155 INFO:teuthology.orchestra.run.smithi107.stdout:5308 read 0x53bd5ae thru 0x53c2f3e (0x5991 bytes) 2024-09-21T15:34:38.155 INFO:teuthology.orchestra.run.smithi107.stdout:5309 read 0xb1769be thru 0xb17aa89 (0x40cc bytes) 2024-09-21T15:34:38.155 INFO:teuthology.orchestra.run.smithi107.stdout:5310 read 0x7fa7da thru 0x7fe126 (0x394d bytes) 2024-09-21T15:34:38.156 INFO:teuthology.orchestra.run.smithi107.stdout:5311 punch from 0x1816729 to 0x1821c06, (0xb4dd bytes) 2024-09-21T15:34:38.157 INFO:teuthology.orchestra.run.smithi107.stdout:5312 write 0x208cb98 thru 0x20929f2 (0x5e5b bytes) 2024-09-21T15:34:38.163 INFO:teuthology.orchestra.run.smithi107.stdout:5313 read 0x3210ca6 thru 0x3211796 (0xaf1 bytes) 2024-09-21T15:34:38.174 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:38.170+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:38.183 INFO:teuthology.orchestra.run.smithi107.stdout:5314 trunc from 0xc42c75f to 0x3dc3861 2024-09-21T15:34:38.241 INFO:teuthology.orchestra.run.smithi107.stdout:5315 write 0xca35998 thru 0xca427e5 (0xce4e bytes) 2024-09-21T15:34:38.248 INFO:teuthology.orchestra.run.smithi107.stdout:5316 writesame 0x94f4799 thru 0x94fbd31 data_size 0x9(0x7599 bytes) 2024-09-21T15:34:38.258 INFO:teuthology.orchestra.run.smithi107.stdout:5317 read 0x9176111 thru 0x9182012 (0xbf02 bytes) 2024-09-21T15:34:38.259 INFO:teuthology.orchestra.run.smithi107.stdout:5318 read 0x3f001b4 thru 0x3f0732c (0x7179 bytes) 2024-09-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:38.260 INFO:teuthology.orchestra.run.smithi107.stdout:5319 write 0x5ed6465 thru 0x5ed7f46 (0x1ae2 bytes) 2024-09-21T15:34:38.267 INFO:teuthology.orchestra.run.smithi107.stdout:5322 write 0x81ab9c6 thru 0x81b0514 (0x4b4f bytes) 2024-09-21T15:34:38.274 INFO:teuthology.orchestra.run.smithi107.stdout:5325 read 0xc9bec3c thru 0xc9c0e2c (0x21f1 bytes) 2024-09-21T15:34:38.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:38.276 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:38.272+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:38.276 INFO:teuthology.orchestra.run.smithi107.stdout:5327 writesame 0x261b883 thru 0x2627d09 data_size 0x1(0xc487 bytes) 2024-09-21T15:34:38.357 INFO:teuthology.orchestra.run.smithi107.stdout:5328 write 0xca62453 thru 0xca63ed6 (0x1a84 bytes) 2024-09-21T15:34:38.360 INFO:teuthology.orchestra.run.smithi107.stdout:5329 write 0x4c837f8 thru 0x4c8adf5 (0x75fe bytes) 2024-09-21T15:34:38.371 INFO:teuthology.orchestra.run.smithi107.stdout:5330 trunc from 0xca63ed7 to 0xe6b09d8 2024-09-21T15:34:38.375 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:38.372+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:38.384 INFO:teuthology.orchestra.run.smithi107.stdout:5331 write 0x5d3072b thru 0x5d32372 (0x1c48 bytes) 2024-09-21T15:34:38.416 INFO:teuthology.orchestra.run.smithi107.stdout:5332 trunc from 0xe6b09d8 to 0x1949e06 2024-09-21T15:34:38.459 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:38.459 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:38.463 INFO:teuthology.orchestra.run.smithi107.stdout:5333 write 0x1a8226b thru 0x1a8bb99 (0x992f bytes) 2024-09-21T15:34:38.467 INFO:teuthology.orchestra.run.smithi107.stdout:5334 read 0xdc938a thru 0xdd66f0 (0xd367 bytes) 2024-09-21T15:34:38.468 INFO:teuthology.orchestra.run.smithi107.stdout:5335 writesame 0x8d6ca1 thru 0x8dddc3 data_size 0x1(0x7123 bytes) 2024-09-21T15:34:38.473 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:38.477 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:38.476+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:38.508 INFO:teuthology.orchestra.run.smithi107.stdout:5336 read 0x160b952 thru 0x160cbff (0x12ae bytes) 2024-09-21T15:34:38.510 INFO:teuthology.orchestra.run.smithi107.stdout:5338 punch from 0x176fbbb to 0x177366a, (0x3aaf bytes) 2024-09-21T15:34:38.510 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.539 INFO:teuthology.orchestra.run.smithi107.stdout:5341 write 0x93912a7 thru 0x93921f3 (0xf4d bytes) 2024-09-21T15:34:38.545 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.545 INFO:teuthology.orchestra.run.smithi107.stdout:5344 read 0x2f05660 thru 0x2f05984 (0x325 bytes) 2024-09-21T15:34:38.554 INFO:teuthology.orchestra.run.smithi107.stdout:5345 trunc from 0x93921f4 to 0xcf324d2 2024-09-21T15:34:38.566 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.566 INFO:teuthology.orchestra.run.smithi107.stdout:5348 write 0xad51d7d thru 0xad5f705 (0xd989 bytes) 2024-09-21T15:34:38.574 INFO:teuthology.orchestra.run.smithi107.stdout:5349 read 0x20b442f thru 0x20c1672 (0xd244 bytes) 2024-09-21T15:34:38.574 INFO:teuthology.orchestra.run.smithi107.stdout:5350 write 0x3b68d8e thru 0x3b6f7f9 (0x6a6c bytes) 2024-09-21T15:34:38.577 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:38.574+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:38.580 INFO:teuthology.orchestra.run.smithi107.stdout:5353 read 0xa0d13ab thru 0xa0dc3a6 (0xaffc bytes) 2024-09-21T15:34:38.581 INFO:teuthology.orchestra.run.smithi107.stdout:5355 write 0x696b36b thru 0x696bd2b (0x9c1 bytes) 2024-09-21T15:34:38.587 INFO:teuthology.orchestra.run.smithi107.stdout:5356 write 0xac4804c thru 0xac51638 (0x95ed bytes) 2024-09-21T15:34:38.605 INFO:teuthology.orchestra.run.smithi107.stdout:5357 trunc from 0xcf324d2 to 0x6af5535 2024-09-21T15:34:38.629 INFO:teuthology.orchestra.run.smithi107.stdout:5358 read 0x9c44b3 thru 0x9d234d (0xde9b bytes) 2024-09-21T15:34:38.636 INFO:teuthology.orchestra.run.smithi107.stdout:5359 trunc from 0x6af5535 to 0x3d331ad 2024-09-21T15:34:38.642 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:38.642 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:5365 read 0x1b0e798 thru 0x1b1cb76 (0xe3df bytes) 2024-09-21T15:34:38.657 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:38.692 INFO:teuthology.orchestra.run.smithi107.stdout:5366 write 0xcd639bd thru 0xcd705c8 (0xcc0c bytes) 2024-09-21T15:34:38.693 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:38.690+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:38.699 INFO:teuthology.orchestra.run.smithi107.stdout:5368 writesame 0x530a95b thru 0x5315f9d data_size 0x67(0xb643 bytes) 2024-09-21T15:34:38.721 INFO:teuthology.orchestra.run.smithi107.stdout:5369 write 0xe003c70 thru 0xe0102d7 (0xc668 bytes) 2024-09-21T15:34:38.724 INFO:teuthology.orchestra.run.smithi107.stdout:5373 read 0xd4d3de2 thru 0xd4e0637 (0xc856 bytes) 2024-09-21T15:34:38.724 INFO:teuthology.orchestra.run.smithi107.stdout:5374 writesame 0x28032f1 thru 0x28106e3 data_size 0x1(0xd3f3 bytes) 2024-09-21T15:34:38.779 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:38.775+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:38.798 INFO:teuthology.orchestra.run.smithi107.stdout:5375 write 0x76489c2 thru 0x76532cd (0xa90c bytes) 2024-09-21T15:34:38.807 INFO:teuthology.orchestra.run.smithi107.stdout:5377 write 0x8a8870e thru 0x8a8b3b4 (0x2ca7 bytes) 2024-09-21T15:34:38.814 INFO:teuthology.orchestra.run.smithi107.stdout:5378 write 0xdda9835 thru 0xddae9f0 (0x51bc bytes) 2024-09-21T15:34:38.817 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:38.817 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:38.821 INFO:teuthology.orchestra.run.smithi107.stdout:5379 writesame 0x708b607 thru 0x7098edb data_size 0x1(0xd8d5 bytes) 2024-09-21T15:34:38.833 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:38.857 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:34:38.880 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:38.875+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:38.890 INFO:teuthology.orchestra.run.smithi107.stdout:5380 read 0x960c637 thru 0x960e40b (0x1dd5 bytes) 2024-09-21T15:34:38.890 INFO:teuthology.orchestra.run.smithi107.stdout:5381 writesame 0x7db177d thru 0x7dbf73d data_size 0x7(0xdfc1 bytes) 2024-09-21T15:34:38.899 INFO:teuthology.orchestra.run.smithi107.stdout:5383 writesame 0x882fcd5 thru 0x883bb6f data_size 0x5(0xbe9b bytes) 2024-09-21T15:34:38.919 INFO:teuthology.orchestra.run.smithi107.stdout:5384 trunc from 0xe0102d8 to 0x45d5270 2024-09-21T15:34:38.958 INFO:teuthology.orchestra.run.smithi107.stdout:5385 write 0x665d03f thru 0x666bae2 (0xeaa4 bytes) 2024-09-21T15:34:38.980 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:38.978+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:38.984 INFO:teuthology.orchestra.run.smithi107.stdout:5386 write 0xc914dc7 thru 0xc918b74 (0x3dae bytes) 2024-09-21T15:34:38.990 INFO:teuthology.orchestra.run.smithi107.stdout:5388 write 0x7f150d9 thru 0x7f21d71 (0xcc99 bytes) 2024-09-21T15:34:38.996 INFO:teuthology.orchestra.run.smithi107.stdout:5389 clone 39 order 20 su 4096 sc 6 2024-09-21T15:34:39.021 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:39.021 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:39.036 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:39.081 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:39.078+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:39.159 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:39.194 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:39.194 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:39.194 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:39.190+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:39.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:39.295 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:39.294+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:39.299 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:39.315 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:39.316 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:39.332 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:39.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:39.348 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:39.368 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:39.369 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:39.383 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:39.384 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:39.379+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:39.388 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:39.389 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:39.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:39.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:39.484 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:39.479+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:39.505 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.135986434, 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.107775258, 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:39.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.91847589, 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:39.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:39.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.787091199, 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:39.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.625993795, 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:39.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.625941248, 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:39.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:39.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:39.533 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.625893123, 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:39.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 102.613496235, 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:39.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:39.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:39.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:39.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:39.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 101.052723253, 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.544 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:39.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.985698887, 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:39.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:39.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.850578756, 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:39.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.809916613, 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.809903396, 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:39.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:39.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.80989058900001, 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:39.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.80985018200001, 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:39.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.809825714, 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:39.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.790603726, 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.579 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.70330014300001, 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:39.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:39.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:39.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:39.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:39.589 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.708614699999998, 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:39.593 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 32.294355353, 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:39.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:39.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:39.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:39.598 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:39.598 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:39.599 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:39.599 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:39.595+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:39.700 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:39.695+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:39.720 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:34:39.732 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:39.732 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:39.732 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:39.732 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:39.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:39.833 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:39.831+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:34:39.900 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:39.898+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:39.919 INFO:teuthology.orchestra.run.smithi107.stdout:leaving image image_client.0-clone38 intact 2024-09-21T15:34:39.923 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:39.923 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:39.923 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:39.923 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:39.923 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:39.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:40.001 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:39.998+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:40.050 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:40.051 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:34:40.070 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-21T15:34:39.836013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":923073830,"num_objects":1380,"num_object_clones":304,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":475,"num_read":200998,"num_read_kb":335758,"num_write":14315,"num_write_kb":310318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":20367099,"num_keys_recovered":127,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11322,"ondisk_log_size":11322,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":484764,"kb_used_data":193032,"kb_used_omap":12,"kb_used_meta":291443,"kb_avail":754489956,"statfs":{"total":773094113280,"available":772597714944,"internally_reserved":0,"allocated":197664768,"data_stored":227399418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":298438201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65149851,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":6,"num_read":3735,"num_read_kb":19011,"num_write":827,"num_write_kb":13852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.038823"},"pg_stats":[{"pgid":"3.1f","version":"105'566","reported_seq":5063,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.652165+0000","last_change":"2024-09-21T15:32:48.729833+0000","last_active":"2024-09-21T15:34:38.652165+0000","last_peered":"2024-09-21T15:34:38.652165+0000","last_clean":"2024-09-21T15:34:38.652165+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:38.652165+0000","last_undegraded":"2024-09-21T15:34:38.652165+0000","last_fullsized":"2024-09-21T15:34:38.652165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027033589999999998,"stat_sum":{"num_bytes":22084322,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":6777,"num_read_kb":7265,"num_write":882,"num_write_kb":2643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"105'392","reported_seq":6794,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.656117+0000","last_change":"2024-09-21T15:33:03.924766+0000","last_active":"2024-09-21T15:34:38.656117+0000","last_peered":"2024-09-21T15:34:38.656117+0000","last_clean":"2024-09-21T15:34:38.656117+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:38.656117+0000","last_undegraded":"2024-09-21T15:34:38.656117+0000","last_fullsized":"2024-09-21T15:34:38.656117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000542823,"stat_sum":{"num_bytes":31764946,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":7,"num_read":8059,"num_read_kb":21249,"num_write":520,"num_write_kb":26402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"104'129","reported_seq":432,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.912673+0000","last_change":"2024-09-21T15:32:48.746252+0000","last_active":"2024-09-21T15:34:35.912673+0000","last_peered":"2024-09-21T15:34:35.912673+0000","last_clean":"2024-09-21T15:34:35.912673+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:35.912673+0000","last_undegraded":"2024-09-21T15:34:35.912673+0000","last_fullsized":"2024-09-21T15:34:35.912673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024024649999999999,"stat_sum":{"num_bytes":33678683,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":264,"num_read_kb":4854,"num_write":184,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"105'226","reported_seq":1375,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.592765+0000","last_change":"2024-09-21T15:32:48.745389+0000","last_active":"2024-09-21T15:34:38.592765+0000","last_peered":"2024-09-21T15:34:38.592765+0000","last_clean":"2024-09-21T15:34:38.592765+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:38.592765+0000","last_undegraded":"2024-09-21T15:34:38.592765+0000","last_fullsized":"2024-09-21T15:34:38.592765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019746849999999999,"stat_sum":{"num_bytes":47697616,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":1251,"num_read_kb":4949,"num_write":234,"num_write_kb":5217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"102'305","reported_seq":9199,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.516540+0000","last_change":"2024-09-21T15:32:48.730622+0000","last_active":"2024-09-21T15:34:36.516540+0000","last_peered":"2024-09-21T15:34:36.516540+0000","last_clean":"2024-09-21T15:34:36.516540+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:36.516540+0000","last_undegraded":"2024-09-21T15:34:36.516540+0000","last_fullsized":"2024-09-21T15:34:36.516540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027480489999999998,"stat_sum":{"num_bytes":17839064,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":9585,"num_read_kb":12345,"num_write":308,"num_write_kb":5281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"105'192","reported_seq":2448,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.631091+0000","last_change":"2024-09-21T15:32:48.725541+0000","last_active":"2024-09-21T15:34:38.631091+0000","last_peered":"2024-09-21T15:34:38.631091+0000","last_clean":"2024-09-21T15:34:38.631091+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:38.631091+0000","last_undegraded":"2024-09-21T15:34:38.631091+0000","last_fullsized":"2024-09-21T15:34:38.631091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093355599999999999,"stat_sum":{"num_bytes":30499508,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":2270,"num_read_kb":13459,"num_write":198,"num_write_kb":14321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"104'553","reported_seq":33257,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.514381+0000","last_change":"2024-09-21T15:32:48.729564+0000","last_active":"2024-09-21T15:34:36.514381+0000","last_peered":"2024-09-21T15:34:36.514381+0000","last_clean":"2024-09-21T15:34:36.514381+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:36.514381+0000","last_undegraded":"2024-09-21T15:34:36.514381+0000","last_fullsized":"2024-09-21T15:34:36.514381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010146319999999999,"stat_sum":{"num_bytes":64439789,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":34606,"num_read_kb":19487,"num_write":790,"num_write_kb":4976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"103'395","reported_seq":921,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.078745+0000","last_change":"2024-09-21T15:33:03.922043+0000","last_active":"2024-09-21T15:34:36.078745+0000","last_peered":"2024-09-21T15:34:36.078745+0000","last_clean":"2024-09-21T15:34:36.078745+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:36.078745+0000","last_undegraded":"2024-09-21T15:34:36.078745+0000","last_fullsized":"2024-09-21T15:34:36.078745+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056718100000000004,"stat_sum":{"num_bytes":17567475,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":2044,"num_read_kb":4684,"num_write":556,"num_write_kb":13126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"105'591","reported_seq":1809,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.368348+0000","last_change":"2024-09-21T15:32:48.730452+0000","last_active":"2024-09-21T15:34:38.368348+0000","last_peered":"2024-09-21T15:34:38.368348+0000","last_clean":"2024-09-21T15:34:38.368348+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:38.368348+0000","last_undegraded":"2024-09-21T15:34:38.368348+0000","last_fullsized":"2024-09-21T15:34:38.368348+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029992410000000001,"stat_sum":{"num_bytes":29991782,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3862,"num_read_kb":11473,"num_write":899,"num_write_kb":7754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"103'292","reported_seq":5764,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.817933+0000","last_change":"2024-09-21T15:32:48.745493+0000","last_active":"2024-09-21T15:34:35.817933+0000","last_peered":"2024-09-21T15:34:35.817933+0000","last_clean":"2024-09-21T15:34:35.817933+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:35.817933+0000","last_undegraded":"2024-09-21T15:34:35.817933+0000","last_fullsized":"2024-09-21T15:34:35.817933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018730019999999999,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6954,"num_read_kb":9218,"num_write":299,"num_write_kb":3708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"105'248","reported_seq":5002,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.546216+0000","last_change":"2024-09-21T15:34:33.804352+0000","last_active":"2024-09-21T15:34:38.546216+0000","last_peered":"2024-09-21T15:34:38.546216+0000","last_clean":"2024-09-21T15:34:38.546216+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:34:38.546216+0000","last_undegraded":"2024-09-21T15:34:38.546216+0000","last_fullsized":"2024-09-21T15:34:38.546216+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071110900000000002,"stat_sum":{"num_bytes":14420493,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":4835,"num_read_kb":10186,"num_write":290,"num_write_kb":11153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"105'541","reported_seq":1680,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.435703+0000","last_change":"2024-09-21T15:34:22.150690+0000","last_active":"2024-09-21T15:34:38.435703+0000","last_peered":"2024-09-21T15:34:38.435703+0000","last_clean":"2024-09-21T15:34:38.435703+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:38.435703+0000","last_undegraded":"2024-09-21T15:34:38.435703+0000","last_fullsized":"2024-09-21T15:34:38.435703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014159559999999999,"stat_sum":{"num_bytes":42190245,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":3557,"num_read_kb":6023,"num_write":687,"num_write_kb":12035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.861922+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:34:35.861922+0000","last_peered":"2024-09-21T15:34:35.861922+0000","last_clean":"2024-09-21T15:34:35.861922+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:34:35.861922+0000","last_undegraded":"2024-09-21T15:34:35.861922+0000","last_fullsized":"2024-09-21T15:34:35.861922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"103'301","reported_seq":3536,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.811527+0000","last_change":"2024-09-21T15:32:48.729931+0000","last_active":"2024-09-21T15:34:35.811527+0000","last_peered":"2024-09-21T15:34:35.811527+0000","last_clean":"2024-09-21T15:34:35.811527+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:35.811527+0000","last_undegraded":"2024-09-21T15:34:35.811527+0000","last_fullsized":"2024-09-21T15:34:35.811527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025642500000000001,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":5368,"num_read_kb":10541,"num_write":281,"num_write_kb":3828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.812472+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:35.812472+0000","last_peered":"2024-09-21T15:34:35.812472+0000","last_clean":"2024-09-21T15:34:35.812472+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:35.812472+0000","last_undegraded":"2024-09-21T15:34:35.812472+0000","last_fullsized":"2024-09-21T15:34:35.812472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"101'181","reported_seq":1217,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.515979+0000","last_change":"2024-09-21T15:33:03.919946+0000","last_active":"2024-09-21T15:34:36.515979+0000","last_peered":"2024-09-21T15:34:36.515979+0000","last_clean":"2024-09-21T15:34:36.515979+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:36.515979+0000","last_undegraded":"2024-09-21T15:34:36.515979+0000","last_fullsized":"2024-09-21T15:34:36.515979+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055978099999999997,"stat_sum":{"num_bytes":37229564,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":1393,"num_read_kb":22596,"num_write":201,"num_write_kb":18044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.813461+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:35.813461+0000","last_peered":"2024-09-21T15:34:35.813461+0000","last_clean":"2024-09-21T15:34:35.813461+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:35.813461+0000","last_undegraded":"2024-09-21T15:34:35.813461+0000","last_fullsized":"2024-09-21T15:34:35.813461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"104'172","reported_seq":738,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.514940+0000","last_change":"2024-09-21T15:33:21.129263+0000","last_active":"2024-09-21T15:34:36.514940+0000","last_peered":"2024-09-21T15:34:36.514940+0000","last_clean":"2024-09-21T15:34:36.514940+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:36.514940+0000","last_undegraded":"2024-09-21T15:34:36.514940+0000","last_fullsized":"2024-09-21T15:34:36.514940+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017633,"stat_sum":{"num_bytes":18384196,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":821,"num_read_kb":4058,"num_write":176,"num_write_kb":2734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.812160+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:35.812160+0000","last_peered":"2024-09-21T15:34:35.812160+0000","last_clean":"2024-09-21T15:34:35.812160+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:35.812160+0000","last_undegraded":"2024-09-21T15:34:35.812160+0000","last_fullsized":"2024-09-21T15:34:35.812160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"104'383","reported_seq":789,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.083064+0000","last_change":"2024-09-21T15:32:48.746703+0000","last_active":"2024-09-21T15:34:36.083064+0000","last_peered":"2024-09-21T15:34:36.083064+0000","last_clean":"2024-09-21T15:34:36.083064+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:36.083064+0000","last_undegraded":"2024-09-21T15:34:36.083064+0000","last_fullsized":"2024-09-21T15:34:36.083064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028282289999999998,"stat_sum":{"num_bytes":13447939,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":1467,"num_read_kb":4790,"num_write":493,"num_write_kb":3979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"104'191","reported_seq":6011,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.514806+0000","last_change":"2024-09-21T15:32:48.725234+0000","last_active":"2024-09-21T15:34:36.514806+0000","last_peered":"2024-09-21T15:34:36.514806+0000","last_clean":"2024-09-21T15:34:36.514806+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:36.514806+0000","last_undegraded":"2024-09-21T15:34:36.514806+0000","last_fullsized":"2024-09-21T15:34:36.514806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063168700000000003,"stat_sum":{"num_bytes":24593565,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":5806,"num_read_kb":12014,"num_write":216,"num_write_kb":4039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":102,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.817951+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:35.817951+0000","last_peered":"2024-09-21T15:34:35.817951+0000","last_clean":"2024-09-21T15:34:35.817951+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:35.817951+0000","last_undegraded":"2024-09-21T15:34:35.817951+0000","last_fullsized":"2024-09-21T15:34:35.817951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"104'681","reported_seq":1827,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.378168+0000","last_change":"2024-09-21T15:33:24.082603+0000","last_active":"2024-09-21T15:34:36.378168+0000","last_peered":"2024-09-21T15:34:36.378168+0000","last_clean":"2024-09-21T15:34:36.378168+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:36.378168+0000","last_undegraded":"2024-09-21T15:34:36.378168+0000","last_fullsized":"2024-09-21T15:34:36.378168+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089245,"stat_sum":{"num_bytes":26067297,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":4416,"num_read_kb":5815,"num_write":967,"num_write_kb":10671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.879825+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:35.879825+0000","last_peered":"2024-09-21T15:34:35.879825+0000","last_clean":"2024-09-21T15:34:35.879825+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:35.879825+0000","last_undegraded":"2024-09-21T15:34:35.879825+0000","last_fullsized":"2024-09-21T15:34:35.879825+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"105'447","reported_seq":2576,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:37.708918+0000","last_change":"2024-09-21T15:34:29.764833+0000","last_active":"2024-09-21T15:34:37.708918+0000","last_peered":"2024-09-21T15:34:37.708918+0000","last_clean":"2024-09-21T15:34:37.708918+0000","last_became_active":"2024-09-21T15:34:29.762828+0000","last_became_peered":"2024-09-21T15:34:29.762828+0000","last_unstale":"2024-09-21T15:34:37.708918+0000","last_undegraded":"2024-09-21T15:34:37.708918+0000","last_fullsized":"2024-09-21T15:34:37.708918+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:17.503095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021225969999999999,"stat_sum":{"num_bytes":25830907,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":11,"num_read":3614,"num_read_kb":10065,"num_write":643,"num_write_kb":12826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.875200+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:36.875200+0000","last_peered":"2024-09-21T15:34:36.875200+0000","last_clean":"2024-09-21T15:34:36.875200+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:36.875200+0000","last_undegraded":"2024-09-21T15:34:36.875200+0000","last_fullsized":"2024-09-21T15:34:36.875200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"104'208","reported_seq":714,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.898108+0000","last_change":"2024-09-21T15:33:03.924419+0000","last_active":"2024-09-21T15:34:35.898108+0000","last_peered":"2024-09-21T15:34:35.898108+0000","last_clean":"2024-09-21T15:34:35.898108+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:35.898108+0000","last_undegraded":"2024-09-21T15:34:35.898108+0000","last_fullsized":"2024-09-21T15:34:35.898108+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000608707,"stat_sum":{"num_bytes":24646899,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":25,"num_read":995,"num_read_kb":3967,"num_write":194,"num_write_kb":1734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":125,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.817785+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:35.817785+0000","last_peered":"2024-09-21T15:34:35.817785+0000","last_clean":"2024-09-21T15:34:35.817785+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:35.817785+0000","last_undegraded":"2024-09-21T15:34:35.817785+0000","last_fullsized":"2024-09-21T15:34:35.817785+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.813178+0000","last_change":"2024-09-21T15:34:33.800946+0000","last_active":"2024-09-21T15:34:35.813178+0000","last_peered":"2024-09-21T15:34:35.813178+0000","last_clean":"2024-09-21T15:34:35.813178+0000","last_became_active":"2024-09-21T15:34:33.799956+0000","last_became_peered":"2024-09-21T15:34:33.799956+0000","last_unstale":"2024-09-21T15:34:35.813178+0000","last_undegraded":"2024-09-21T15:34:35.813178+0000","last_fullsized":"2024-09-21T15:34:35.813178+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:57.645467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"104'537","reported_seq":1462,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.079877+0000","last_change":"2024-09-21T15:34:29.765868+0000","last_active":"2024-09-21T15:34:36.079877+0000","last_peered":"2024-09-21T15:34:36.079877+0000","last_clean":"2024-09-21T15:34:36.079877+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:34:36.079877+0000","last_undegraded":"2024-09-21T15:34:36.079877+0000","last_fullsized":"2024-09-21T15:34:36.079877+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037806299999999999,"stat_sum":{"num_bytes":12931104,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2898,"num_read_kb":7080,"num_write":766,"num_write_kb":5818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'454","reported_seq":1511,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.970850+0000","last_change":"2024-09-21T15:33:21.111697+0000","last_active":"2024-09-21T15:34:35.970850+0000","last_peered":"2024-09-21T15:34:35.970850+0000","last_clean":"2024-09-21T15:34:35.970850+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:35.970850+0000","last_undegraded":"2024-09-21T15:34:35.970850+0000","last_fullsized":"2024-09-21T15:34:35.970850+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001411236,"stat_sum":{"num_bytes":24522187,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":3124,"num_read_kb":7564,"num_write":615,"num_write_kb":3711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"104'267","reported_seq":4761,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.083945+0000","last_change":"2024-09-21T15:32:48.757417+0000","last_active":"2024-09-21T15:34:36.083945+0000","last_peered":"2024-09-21T15:34:36.083945+0000","last_clean":"2024-09-21T15:34:36.083945+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:36.083945+0000","last_undegraded":"2024-09-21T15:34:36.083945+0000","last_fullsized":"2024-09-21T15:34:36.083945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019449549999999999,"stat_sum":{"num_bytes":21750935,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4752,"num_read_kb":13732,"num_write":291,"num_write_kb":5540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"105'309","reported_seq":4705,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.608923+0000","last_change":"2024-09-21T15:33:03.919900+0000","last_active":"2024-09-21T15:34:38.608923+0000","last_peered":"2024-09-21T15:34:38.608923+0000","last_clean":"2024-09-21T15:34:38.608923+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:38.608923+0000","last_undegraded":"2024-09-21T15:34:38.608923+0000","last_fullsized":"2024-09-21T15:34:38.608923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038486,"stat_sum":{"num_bytes":33161860,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":6065,"num_read_kb":11427,"num_write":334,"num_write_kb":9642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"104'231","reported_seq":1970,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.515602+0000","last_change":"2024-09-21T15:32:48.746159+0000","last_active":"2024-09-21T15:34:36.515602+0000","last_peered":"2024-09-21T15:34:36.515602+0000","last_clean":"2024-09-21T15:34:36.515602+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:36.515602+0000","last_undegraded":"2024-09-21T15:34:36.515602+0000","last_fullsized":"2024-09-21T15:34:36.515602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022814659999999998,"stat_sum":{"num_bytes":37448049,"num_objects":45,"num_object_clones":17,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":1824,"num_read_kb":8363,"num_write":237,"num_write_kb":21357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'332","reported_seq":17301,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.813467+0000","last_change":"2024-09-21T15:32:48.962919+0000","last_active":"2024-09-21T15:34:35.813467+0000","last_peered":"2024-09-21T15:34:35.813467+0000","last_clean":"2024-09-21T15:34:35.813467+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:35.813467+0000","last_undegraded":"2024-09-21T15:34:35.813467+0000","last_fullsized":"2024-09-21T15:34:35.813467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081583999999999997,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17970,"num_read_kb":4982,"num_write":485,"num_write_kb":10274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"105'297","reported_seq":33631,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.581745+0000","last_change":"2024-09-21T15:34:30.767437+0000","last_active":"2024-09-21T15:34:38.581745+0000","last_peered":"2024-09-21T15:34:38.581745+0000","last_clean":"2024-09-21T15:34:38.581745+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:34:38.581745+0000","last_undegraded":"2024-09-21T15:34:38.581745+0000","last_fullsized":"2024-09-21T15:34:38.581745+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012772040000000001,"stat_sum":{"num_bytes":54556822,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":33834,"num_read_kb":32923,"num_write":331,"num_write_kb":20969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"100'303","reported_seq":1586,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.082366+0000","last_change":"2024-09-21T15:34:33.806455+0000","last_active":"2024-09-21T15:34:36.082366+0000","last_peered":"2024-09-21T15:34:36.082366+0000","last_clean":"2024-09-21T15:34:36.082366+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:34:36.082366+0000","last_undegraded":"2024-09-21T15:34:36.082366+0000","last_fullsized":"2024-09-21T15:34:36.082366+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011368179999999999,"stat_sum":{"num_bytes":30043907,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":2338,"num_read_kb":6349,"num_write":346,"num_write_kb":3654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"92'224","reported_seq":736,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:35.813403+0000","last_change":"2024-09-21T15:33:49.069411+0000","last_active":"2024-09-21T15:34:35.813403+0000","last_peered":"2024-09-21T15:34:35.813403+0000","last_clean":"2024-09-21T15:34:35.813403+0000","last_became_active":"2024-09-21T15:33:48.931710+0000","last_became_peered":"2024-09-21T15:33:48.931710+0000","last_unstale":"2024-09-21T15:34:35.813403+0000","last_undegraded":"2024-09-21T15:34:35.813403+0000","last_fullsized":"2024-09-21T15:34:35.813403+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:45:41.605564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":17913535,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":976,"num_read_kb":6250,"num_write":213,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":798967,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.11","version":"104'433","reported_seq":5967,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-21T15:34:36.516850+0000","last_change":"2024-09-21T15:32:48.962096+0000","last_active":"2024-09-21T15:34:36.516850+0000","last_peered":"2024-09-21T15:34:36.516850+0000","last_clean":"2024-09-21T15:34:36.516850+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:36.516850+0000","last_undegraded":"2024-09-21T15:34:36.516850+0000","last_fullsized":"2024-09-21T15:34:36.516850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059123200000000002,"stat_sum":{"num_bytes":27596605,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":8000,"num_read_kb":20401,"num_write":566,"num_write_kb":18101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"105'479","reported_seq":7161,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.275577+0000","last_change":"2024-09-21T15:34:33.799675+0000","last_active":"2024-09-21T15:34:38.275577+0000","last_peered":"2024-09-21T15:34:38.275577+0000","last_clean":"2024-09-21T15:34:38.275577+0000","last_became_active":"2024-09-21T15:34:33.798598+0000","last_became_peered":"2024-09-21T15:34:33.798598+0000","last_unstale":"2024-09-21T15:34:38.275577+0000","last_undegraded":"2024-09-21T15:34:38.275577+0000","last_fullsized":"2024-09-21T15:34:38.275577+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:18.729091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001874988,"stat_sum":{"num_bytes":21295696,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":7045,"num_read_kb":12135,"num_write":625,"num_write_kb":3479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2138377,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"105'342","reported_seq":1865,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T15:34:38.510161+0000","last_change":"2024-09-21T15:32:48.962257+0000","last_active":"2024-09-21T15:34:38.510161+0000","last_peered":"2024-09-21T15:34:38.510161+0000","last_clean":"2024-09-21T15:34:38.510161+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:38.510161+0000","last_undegraded":"2024-09-21T15:34:38.510161+0000","last_fullsized":"2024-09-21T15:34:38.510161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067303100000000004,"stat_sum":{"num_bytes":40145065,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":4112,"num_read_kb":5414,"num_write":327,"num_write_kb":11068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":922483443,"num_objects":1376,"num_object_clones":304,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":475,"num_read":200882,"num_read_kb":335658,"num_write":14154,"num_write_kb":307900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":19186363,"num_keys_recovered":127,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186662912,"data_stored":221742884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11202,"ondisk_log_size":11202,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64696,"kb_used_data":33576,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94307144,"statfs":{"total":96636764160,"available":96570515456,"internally_reserved":0,"allocated":34381824,"data_stored":38777889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.51500000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":1.0129999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":1.1339999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.038},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":0.92400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":12120,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":12410880,"data_stored":13707786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.40699999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.45400000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.439},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.375}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.57999999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.497},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":0.56000000000000005}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.626},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":24276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":24858624,"data_stored":29243406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52200000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":1.0249999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.95199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68220,"kb_used_data":37048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94303620,"statfs":{"total":96636764160,"available":96566906880,"internally_reserved":0,"allocated":37937152,"data_stored":41258060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.53700000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.91800000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.879},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":0.40799999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.71599999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.59399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81252,"kb_used_data":37844,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94290588,"statfs":{"total":96636764160,"available":96553562112,"internally_reserved":0,"allocated":38752256,"data_stored":47648109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44431816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.5},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":0.56000000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.621},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.51900000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.70199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42268,"kb_used_data":6992,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329572,"statfs":{"total":96636764160,"available":96593481728,"internally_reserved":0,"allocated":7159808,"data_stored":7516526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.34399999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.72799999999999998,"15min":0.75800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.613,"15min":1.613},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.36599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.159,"15min":1.159},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90100000000000002,"15min":0.91800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1919999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.536,"15min":1.536},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1699999999999999,"5min":1.52,"15min":1.52},"last":0.69999999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.94399999999999995,"15min":0.96999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0860000000000001,"5min":1.617,"15min":1.617},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.95199999999999996,"15min":0.97799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.995,"5min":1.554,"15min":1.554},"last":0.78600000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1200000000000001,"5min":1.669,"15min":1.669},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2390000000000001,"5min":1.589,"15min":1.589},"last":0.74099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56028,"kb_used_data":16660,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94315812,"statfs":{"total":96636764160,"available":96579391488,"internally_reserved":0,"allocated":17059840,"data_stored":19313498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.501}]},{"osd":3,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.0669999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.81999999999999995,"15min":0.79600000000000004},"min":{"1min":0.55000000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.55900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2390000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2729999999999999,"5min":1.514,"15min":1.514},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.67200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.891,"15min":1.891},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.94099999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3480000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.026}]},{"osd":7,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.98899999999999999,"15min":1.004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3919999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.95699999999999996,"15min":0.93999999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69580,"kb_used_data":24516,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94302260,"statfs":{"total":96636764160,"available":96565514240,"internally_reserved":0,"allocated":25104384,"data_stored":29934144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.28599999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0329999999999999,"5min":1.397,"15min":1.397},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.4870000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.016,"5min":1.23,"15min":1.23},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.038,"5min":1.133,"15min":1.133},"last":0.57499999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.0229999999999997,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3120000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.48599999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.91200000000000003,"15min":0.92600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2110000000000001,"5min":1.867,"15min":1.867},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.95499999999999996,"15min":0.93799999999999994},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.72299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0049999999999999,"15min":1.0169999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.464,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.55500000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2649999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.79500000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":0.97899999999999998},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.427,"5min":1.804,"15min":1.804},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0169999999999999,"15min":1.0009999999999999},"min":{"1min":0.58899999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23203840,"data_stored":28713019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15896576,"data_stored":18811962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5967872,"data_stored":6987490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37552128,"data_stored":47113496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36794368,"data_stored":40772210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23502848,"data_stored":28568042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11161600,"data_stored":13126514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32583680,"data_stored":37650151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:34:40.072 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T15:34:40.073 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:40.073 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T15:34:40.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T15:34:40.101 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:40.099+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.113 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833692671000001, 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.117 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833668611999997, 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:40.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:40.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:40.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.121 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833644649, 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:40.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833620883000002, 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833597241, 2024-09-21T15:34:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833571687999999, 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833527472999997, 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833498763999998, 2024-09-21T15:34:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:40.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:40.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833467220999999, 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833440148999998, 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:40.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:40.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.833414216000001, 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:40.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:40.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:40.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.157 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.832879273000003, 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:40.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:40.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:40.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.161 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.832249605000001, 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:40.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:40.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828338041999999, 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.166 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.169 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828291544000002, 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828242375000002, 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.174 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.177 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828194527000001, 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.178 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.181 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828146138000001, 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.182 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.185 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.828065217000002, 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.186 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.824466854000001, 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:40.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.193 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:40.194 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-21T15:34:40.194 INFO:teuthology.orchestra.run.smithi107.stdout:5391 writesame 0x1f0f9c0 thru 0x1f164b0 data_size 0x1(0x6af1 bytes) 2024-09-21T15:34:40.195 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:40.202 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:40.198+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:40.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T15:34:40.332 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:40.330+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:40.370 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:40.370 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:40.370 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:40.370 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:40.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:40.403 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:40.398+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:40.490 INFO:teuthology.orchestra.run.smithi107.stdout:5392 clone 40 order 25 su 16777216 sc 12 2024-09-21T15:34:40.503 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:40.502+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:40.505 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T15:34:40.519 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:40.519 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:40.519 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:40.519 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:40.519 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:40.533 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:40.604 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:40.602+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:40.649 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.369137334999998, 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.650 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.369078774000002, 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.654 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.369008059000002, 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.658 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368958423999999, 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.662 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368890655999998, 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.666 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368779783000001, 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.670 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368685816000003, 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368637691000004, 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.678 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368568013000001, 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.682 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368518621, 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.686 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368471751999998, 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.690 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368424656999998, 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.694 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:34:40.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:34:40.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:34:40.697 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368375618999998, 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.698 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368307870999999, 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.702 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368259191999996, 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:34:40.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.368148908999999, 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:40.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:34:40.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:34:40.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.365900451999998, 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:40.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:34:40.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.365830701, 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.365741049999997, 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:34:40.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:34:40.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 33.365461267999997, 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.729 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:40.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T15:34:40.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:40.731 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:40.727+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:40.831 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:40.827+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:40.902 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:40.902 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:40.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:172540 3.1e 3:7b74e608:::rbd_header.1191eefab886:head [call rbd.get_stripe_unit_count in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:40.903251+0000", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 0.00018406299999999999, 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000220845, 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "reached pg", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 172540 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:40.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903251+0000", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903251+0000", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903250+0000", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:40.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903258+0000", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.9049999999999997e-06 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903273+0000", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5157000000000001e-05 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903281+0000", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7430000000000002e-06 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:40.903390+0000", 2024-09-21T15:34:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00010897500000000001 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 1 2024-09-21T15:34:40.906 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:40.918 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:40.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T15:34:40.958 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone39 2024-09-21T15:34:40.965 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:40.963+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:41.032 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:41.030+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:41.101 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:41.101 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:41.101 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:41.101 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:41.101 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:41.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-21T15:34:41.115 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:41.165 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:41.163+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:41.234 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:41.230+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:41.266 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:41.266 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:41.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:41.266 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:41.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.606854970000001, 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.270 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.603004388000002, 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.271 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:34:41.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.602979196, 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.275 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:34:41.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:34:41.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.599003174000003, 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.279 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.598977152000003, 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:34:41.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:41.287 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.598953598999998, 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:34:41.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:34:41.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.291 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.598918757, 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.594937235000003, 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.299 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.594829857999997, 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.594803288999998, 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.594775712999997, 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:41.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:34:41.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:34:41.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.311 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.594752202999999, 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:41.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:34:41.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.591050576999997, 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:34:41.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.591020823999997, 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.320 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:41.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.590993755, 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.590970419000001, 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.590901815000002, 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:34:41.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.590873074999998, 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:34:41.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:34:41.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.586960091999998, 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:34:41.340 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 32.586905020000003, 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:34:41.344 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:41.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:34:41.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:34:41.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:41.348 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:41.350 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-21T15:34:41.350 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:41.346+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:41.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:41.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:41.449 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:41.447+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:41.550 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:41.546+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:41.554 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:41.554 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:41.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T15:34:41.570 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:41.606 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:41.606 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:41.626 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:41.626 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:41.642 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:41.651 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:41.646+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:41.661 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:41.682 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:41.682 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:41.700 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:41.700 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:41.717 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:41.718 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:41.738 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:41.738 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:41.752 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:41.747+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:41.776 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:41.776 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:41.794 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:41.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:34:41.881 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:41.879+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:41.954 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:41.951+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:41.977 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:41.977 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:41.996 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:42.054 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:42.050+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:42.118 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.118 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:42.184 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:42.182+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:42.256 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:42.254+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:42.292 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.292 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.304 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:42.356 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:42.355+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:42.457 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:42.454+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:42.576 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:42.574+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:42.640 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.640 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:42.700 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:42.698+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:42.760 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:42.758+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:42.826 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.826 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:42.856 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:34:42.875 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:42.874+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:42.962 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:42.958+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:42.975 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:42.975 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:42.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:43.080 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:43.078+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:43.149 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:43.149 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:43.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:43.164 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:43.162+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:43.264 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:43.259+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:43.330 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:43.330 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:43.344 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:43.365 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:43.363+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:43.466 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:43.462+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:43.471 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:43.472 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:43.486 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:43.567 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:43.563+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:43.652 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:43.653 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:43.668 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:43.666+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:43.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:43.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:43.768 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:43.767+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:43.841 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:43.841 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:43.858 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:43.905 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:43.906 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:43.906 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:43.902+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:43.923 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:43.923 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:43.940 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:43.940 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:43.956 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:43.970 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:43.966+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:43.977 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:43.977 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:43.987 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.61935225400001, 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:43.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:34:43.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:34:43.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:34:43.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.993 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.59114107800001, 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:34:43.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:34:43.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:34:43.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 110.40184171, 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:43.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:34:43.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:34:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:34:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:34:44.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.002 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 110.27045701900001, 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:34:44.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.109359615, 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:34:44.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:34:44.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.10930706800001, 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:34:44.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:34:44.017 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.109258943, 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:34:44.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:34:44.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 107.096862055, 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.023 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.536089073, 2024-09-21T15:34:44.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.033 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.469064707, 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.33394457599999, 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:34:44.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:44.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:34:44.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.293282433, 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:34:44.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:44.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:34:44.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:34:44.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.293269216, 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:44.048 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:34:44.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.29325640899999, 2024-09-21T15:34:44.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:34:44.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:34:44.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.29321600199999, 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:44.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.293191534, 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.273969546, 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 105.186665963, 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:34:44.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:34:44.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 37.191980520000001, 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:34:44.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 36.777721173000003, 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:34:44.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:44.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:44.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:44.083 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:44.084 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:44.084 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:44.085 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:44.085 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:44.086 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:44.086 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:44.086 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:44.087 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:44.079+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:44.185 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:44.182+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:44.228 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:44.229 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:44.229 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:44.229 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:44.243 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:44.286 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:44.282+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:44.377 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:44.378 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:44.378 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:44.378 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:44.378 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:44.387 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:44.382+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:44.392 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:44.488 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:44.483+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:44.534 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256900612999999, 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.535 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:44.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256876554000002, 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256852590999998, 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256828824999999, 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256805182999997, 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256779629999997, 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256735415000001, 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256706706000003, 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256675162999997, 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256648091000002, 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256622157999999, 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.256087215000001, 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.255457546999999, 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251545984000003, 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251499486, 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251450317, 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251402468999999, 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251354079999999, 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.251273159, 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.247674795999998, 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:44.620 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:44.621 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:44.622 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:44.618+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:44.722 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:44.718+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:44.787 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:44.787 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:44.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:44.823 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:44.818+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:44.876 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:34:44.918 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:44.919 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:44.919 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:44.919 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:44.919 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:44.924 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:44.919+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:44.934 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:45.025 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:45.022+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:45.101 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.820254773999999, 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:34:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.820196213000003, 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.820125498000003, 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.820075863, 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:45.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:34:45.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.119 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.820008094999999, 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:34:45.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819897222000002, 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.124 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:34:45.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.128 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819803254999997, 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:34:45.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.132 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819755129999997, 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:45.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819685452000002, 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819636060000001, 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819589190999999, 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.145 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.146 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:34:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:34:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819542095999999, 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:34:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:45.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819493057999999, 2024-09-21T15:34:45.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:34:45.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.157 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.81942531, 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:45.161 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819376630999997, 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.163 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.819266347999999, 2024-09-21T15:34:45.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:34:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.817017890999999, 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.169 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:45.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:34:45.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.816948140000001, 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.173 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:34:45.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.816858488999998, 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.177 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:34:45.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.816578706999998, 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.181 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.185 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:45.186 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:45.187 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:45.182+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:45.287 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:45.283+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:45.313 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:45.313 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:45.314 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:45.314 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:45.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:45.417 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:45.415+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:45.489 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:45.486+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:45.526 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:45.526 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:45.540 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:45.590 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:45.586+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:45.692 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:45.690+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:45.714 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:45.714 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:45.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.054837997, 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.715 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:34:45.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:34:45.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.050987415000002, 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:34:45.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:45.723 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.050962222999999, 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:34:45.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:34:45.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.727 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.046986201000003, 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:45.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:34:45.731 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.046960179000003, 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.732 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:34:45.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.733 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.736 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.046936625999997, 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:45.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:34:45.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:34:45.740 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.046901783999999, 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:34:45.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:34:45.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.042920262000003, 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.042812884999996, 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.749 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:34:45.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:34:45.752 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.042786315999997, 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.753 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.042758739999996, 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.042735229999998, 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.761 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:34:45.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.039033603999997, 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.765 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:34:45.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.039003850999997, 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.769 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:34:45.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.038976781999999, 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.773 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:45.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:34:45.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.038953446000001, 2024-09-21T15:34:45.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:34:45.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.038884842000002, 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:34:45.781 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:34:45.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:34:45.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.038856101999997, 2024-09-21T15:34:45.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:34:45.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.789 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.034943118999998, 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:34:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:45.793 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 37.034888047000003, 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:45.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:45.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:45.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.797 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:45.798 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:45.798 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:45.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:45.799 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:45.794+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:45.873 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:34:45.899 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:45.895+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:45.942 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:45.942 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:45.942 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:45.943 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:45.956 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:46.005 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:46.003+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:46.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:46.131 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:46.130+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:46.137 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:46.137 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:46.137 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:46.137 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:46.137 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:46.152 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:46.202 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:46.199+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:46.242 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:46.242 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:46.263 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:46.263 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:46.269 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.491588215999997, 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.270 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:46.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:46.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.491210225000003, 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.274 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.490990969999999, 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:46.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.490593807000003, 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.490190019000003, 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.286 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:46.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:46.290 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.489734257000002, 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:46.294 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.482507710999997, 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.482141423000002, 2024-09-21T15:34:46.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.481429579999997, 2024-09-21T15:34:46.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.306 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.478169033, 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.310 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.477728814000002, 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:46.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.477417391000003, 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:46.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.318 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.452489782999997, 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.319 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.451860127000003, 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.323 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:46.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:46.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.451455664000001, 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.327 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:46.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.331 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.450917271000002, 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:46.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:46.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.450390253000002, 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.339 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.449968527999999, 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:46.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:46.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:46.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.449553125999998, 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.344 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 51.449002698999998, 2024-09-21T15:34:46.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:46.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:46.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:46.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:46.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:46.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:46.355 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:46.356 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:46.357 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:46.357 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:46.358 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:46.358 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:46.355+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:46.358 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:46.359 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:46.359 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:46.359 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:46.359 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:46.385 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:46.386 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:46.406 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:46.406 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:46.457 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:46.455+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:46.480 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:46.480 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:46.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:46.519 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-21T15:34:46.558 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:46.555+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:46.617 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:46.617 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:46.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:46.688 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:46.686+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:46.759 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:46.754+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:46.769 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:46.770 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:46.770 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T15:34:46.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-21T15:34:46.809 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:46.809 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:46.826 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:46.860 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:46.859+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:46.879 INFO:teuthology.orchestra.run.smithi107.stdout:5393 writesame 0x31ef51c thru 0x31f8b33 data_size 0xc(0x9618 bytes) 2024-09-21T15:34:46.936 INFO:teuthology.orchestra.run.smithi107.stdout:5395 trunc from 0xc918b75 to 0xadfe6d1 2024-09-21T15:34:46.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T15:34:46.961 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:46.959+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:46.997 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:46.997 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:47.062 INFO:teuthology.orchestra.run.smithi107.stdout:5396 trunc from 0xadfe6d1 to 0x5f6b1c7 2024-09-21T15:34:47.062 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:47.059+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:47.092 INFO:teuthology.orchestra.run.smithi107.stdout:5397 writesame 0xa51a678 thru 0xa52a01a data_size 0x1(0xf9a3 bytes) 2024-09-21T15:34:47.092 INFO:teuthology.orchestra.run.smithi107.stdout:5398 read 0x25ffb65 thru 0x2602767 (0x2c03 bytes) 2024-09-21T15:34:47.093 INFO:teuthology.orchestra.run.smithi107.stdout:5399 punch from 0x374b957 to 0x374f72f, (0x3dd8 bytes) 2024-09-21T15:34:47.099 INFO:teuthology.orchestra.run.smithi107.stdout:5401 trunc from 0xa52a01b to 0x575500e 2024-09-21T15:34:47.119 INFO:teuthology.orchestra.run.smithi107.stdout:5402 trunc from 0x575500e to 0xc96a812 2024-09-21T15:34:47.125 INFO:teuthology.orchestra.run.smithi107.stdout:5403 writesame 0x14b3d50 thru 0x14bf13d data_size 0x1b(0xb3ee bytes) 2024-09-21T15:34:47.162 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:47.158+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:47.176 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T15:34:47.201 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.201 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.214 INFO:teuthology.orchestra.run.smithi107.stdout:5404 write 0xd1500aa thru 0xd15e995 (0xe8ec bytes) 2024-09-21T15:34:47.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:47.217 INFO:teuthology.orchestra.run.smithi107.stdout:5405 write 0x64daf7a thru 0x64e087d (0x5904 bytes) 2024-09-21T15:34:47.223 INFO:teuthology.orchestra.run.smithi107.stdout:5406 read 0x42a211a thru 0x42a6e7d (0x4d64 bytes) 2024-09-21T15:34:47.223 INFO:teuthology.orchestra.run.smithi107.stdout:5407 writesame 0x32f91c thru 0x3324dd data_size 0x3(0x2bc2 bytes) 2024-09-21T15:34:47.227 INFO:teuthology.orchestra.run.smithi107.stdout:5409 write 0x8f5194 thru 0x9000c5 (0xaf32 bytes) 2024-09-21T15:34:47.230 INFO:teuthology.orchestra.run.smithi107.stdout:5413 write 0x1047a0e thru 0x1052f81 (0xb574 bytes) 2024-09-21T15:34:47.233 INFO:teuthology.orchestra.run.smithi107.stdout:5414 read 0x992e91 thru 0x9a2475 (0xf5e5 bytes) 2024-09-21T15:34:47.243 INFO:teuthology.orchestra.run.smithi107.stdout:5415 write 0xed0464e thru 0xed1021d (0xbbd0 bytes) 2024-09-21T15:34:47.264 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:47.259+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:47.282 INFO:teuthology.orchestra.run.smithi107.stdout:5416 read 0x9f65cfe thru 0x9f69174 (0x3477 bytes) 2024-09-21T15:34:47.282 INFO:teuthology.orchestra.run.smithi107.stdout:5417 writesame 0xae9374a thru 0xae9581f data_size 0x9(0x20d6 bytes) 2024-09-21T15:34:47.288 INFO:teuthology.orchestra.run.smithi107.stdout:5418 write 0xc7c2485 thru 0xc7c680e (0x438a bytes) 2024-09-21T15:34:47.290 INFO:teuthology.orchestra.run.smithi107.stdout:5419 read 0x2159fe thru 0x2195dd (0x3be0 bytes) 2024-09-21T15:34:47.291 INFO:teuthology.orchestra.run.smithi107.stdout:5420 read 0xe8fd1c2 thru 0xe90afc2 (0xde01 bytes) 2024-09-21T15:34:47.292 INFO:teuthology.orchestra.run.smithi107.stdout:5421 read 0xcc63401 thru 0xcc6e04c (0xac4c bytes) 2024-09-21T15:34:47.306 INFO:teuthology.orchestra.run.smithi107.stdout:5422 trunc from 0xed1021e to 0x2d03887 2024-09-21T15:34:47.334 INFO:teuthology.orchestra.run.smithi107.stdout:5423 read 0x2437223 thru 0x2444379 (0xd157 bytes) 2024-09-21T15:34:47.335 INFO:teuthology.orchestra.run.smithi107.stdout:5424 punch from 0x17d920e to 0x17e09b1, (0x77a3 bytes) 2024-09-21T15:34:47.335 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.335 INFO:teuthology.orchestra.run.smithi107.stdout:5426 read 0xc23ec5 thru 0xc2d7e9 (0x9925 bytes) 2024-09-21T15:34:47.347 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.347 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.348 INFO:teuthology.orchestra.run.smithi107.stdout:5429 writesame 0x75e9daa thru 0x75eb8b0 data_size 0x11(0x1b07 bytes) 2024-09-21T15:34:47.348 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.356 INFO:teuthology.orchestra.run.smithi107.stdout:5431 trunc from 0x75eb8b1 to 0xe4e9b8b 2024-09-21T15:34:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:5433 read 0xd278bd4 thru 0xd28232d (0x975a bytes) 2024-09-21T15:34:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:5434 read 0x48d5298 thru 0x48d68ce (0x1637 bytes) 2024-09-21T15:34:47.365 INFO:teuthology.orchestra.run.smithi107.stdout:5435 write 0x85e123c thru 0x85e7d64 (0x6b29 bytes) 2024-09-21T15:34:47.365 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:47.362+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:47.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:47.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T15:34:47.377 INFO:teuthology.orchestra.run.smithi107.stdout:5436 trunc from 0xe4e9b8b to 0x848bf57 2024-09-21T15:34:47.389 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.396 INFO:teuthology.orchestra.run.smithi107.stdout:5438 write 0x94ecb09 thru 0x94fa66a (0xdb62 bytes) 2024-09-21T15:34:47.404 INFO:teuthology.orchestra.run.smithi107.stdout:5439 write 0x46a3331 thru 0x46b0d72 (0xda42 bytes) 2024-09-21T15:34:47.410 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.421 INFO:teuthology.orchestra.run.smithi107.stdout:5441 trunc from 0x94fa66b to 0x3bd90b 2024-09-21T15:34:47.466 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:47.463+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:47.481 INFO:teuthology.orchestra.run.smithi107.stdout:5443 write 0xd09f0ae thru 0xd0a25b8 (0x350b bytes) 2024-09-21T15:34:47.487 INFO:teuthology.orchestra.run.smithi107.stdout:5445 punch from 0x467dc1 to 0x46fb5d, (0x7d9c bytes) 2024-09-21T15:34:47.487 INFO:teuthology.orchestra.run.smithi107.stdout:5446 punch from 0x5c8b900 to 0x5c8d566, (0x1c66 bytes) 2024-09-21T15:34:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:5447 trunc from 0xd0a25b9 to 0x132c2fe 2024-09-21T15:34:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:47.525 INFO:teuthology.orchestra.run.smithi107.stdout:5448 trunc from 0x132c2fe to 0xd1064c9 2024-09-21T15:34:47.532 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.533 INFO:teuthology.orchestra.run.smithi107.stdout:5450 write 0x3ed0a67 thru 0x3ee02e7 (0xf881 bytes) 2024-09-21T15:34:47.538 INFO:teuthology.orchestra.run.smithi107.stdout:5451 write 0x47f9315 thru 0x47f9f51 (0xc3d bytes) 2024-09-21T15:34:47.544 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T15:34:47.545 INFO:teuthology.orchestra.run.smithi107.stdout:5452 punch from 0xa54cafb to 0xa5508ed, (0x3df2 bytes) 2024-09-21T15:34:47.546 INFO:teuthology.orchestra.run.smithi107.stdout:5453 trunc from 0xd1064c9 to 0xe8a68cb 2024-09-21T15:34:47.553 INFO:teuthology.orchestra.run.smithi107.stdout:5454 write 0x23c7271 thru 0x23d6a4b (0xf7db bytes) 2024-09-21T15:34:47.559 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.559 INFO:teuthology.orchestra.run.smithi107.stdout:5459 punch from 0xaab33ab to 0xaab7929, (0x457e bytes) 2024-09-21T15:34:47.559 INFO:teuthology.orchestra.run.smithi107.stdout:5460 punch from 0xc474a04 to 0xc47967b, (0x4c77 bytes) 2024-09-21T15:34:47.560 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.560 INFO:teuthology.orchestra.run.smithi107.stdout:5466 writesame 0xd192e38 thru 0xd1993f3 data_size 0x22(0x65bc bytes) 2024-09-21T15:34:47.563 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.563 INFO:teuthology.orchestra.run.smithi107.stdout:5468 punch from 0x552ea to 0x65062, (0xfd78 bytes) 2024-09-21T15:34:47.563 INFO:teuthology.orchestra.run.smithi107.stdout:5469 read 0xdffe004 thru 0xe003673 (0x5670 bytes) 2024-09-21T15:34:47.563 INFO:teuthology.orchestra.run.smithi107.stdout:5470 writesame 0x1ecd1bd thru 0x1ed9e36 data_size 0x7(0xcc7a bytes) 2024-09-21T15:34:47.567 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:47.563+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:47.568 INFO:teuthology.orchestra.run.smithi107.stdout:5471 read 0x8a0fbaf thru 0x8a1b98a (0xbddc bytes) 2024-09-21T15:34:47.568 INFO:teuthology.orchestra.run.smithi107.stdout:5472 read 0x209e52 thru 0x209f50 (0xff bytes) 2024-09-21T15:34:47.572 INFO:teuthology.orchestra.run.smithi107.stdout:5473 trunc from 0xe8a68cb to 0xb611c8f 2024-09-21T15:34:47.582 INFO:teuthology.orchestra.run.smithi107.stdout:5476 read 0x5ac8baa thru 0x5ad5567 (0xc9be bytes) 2024-09-21T15:34:47.592 INFO:teuthology.orchestra.run.smithi107.stdout:5477 write 0xdc4b4df thru 0xdc53158 (0x7c7a bytes) 2024-09-21T15:34:47.595 INFO:teuthology.orchestra.run.smithi107.stdout:5478 writesame 0x17175b1 thru 0x17263a7 data_size 0x1(0xedf7 bytes) 2024-09-21T15:34:47.625 INFO:teuthology.orchestra.run.smithi107.stdout:5479 punch from 0x345afbd to 0x3461281, (0x62c4 bytes) 2024-09-21T15:34:47.626 INFO:teuthology.orchestra.run.smithi107.stdout:5480 writesame 0x81587bf thru 0x815cdba data_size 0x6(0x45fc bytes) 2024-09-21T15:34:47.631 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.631 INFO:teuthology.orchestra.run.smithi107.stdout:5482 write 0x82b1a9e thru 0x82b517a (0x36dd bytes) 2024-09-21T15:34:47.633 INFO:teuthology.orchestra.run.smithi107.stdout:5484 read 0x15d2013 thru 0x15d5016 (0x3004 bytes) 2024-09-21T15:34:47.634 INFO:teuthology.orchestra.run.smithi107.stdout:5485 write 0x2d2055 thru 0x2da973 (0x891f bytes) 2024-09-21T15:34:47.641 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.641 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.645 INFO:teuthology.orchestra.run.smithi107.stdout:5487 write 0xe2971ea thru 0xe29ee31 (0x7c48 bytes) 2024-09-21T15:34:47.648 INFO:teuthology.orchestra.run.smithi107.stdout:5488 punch from 0x1b5039e to 0x1b5da4c, (0xd6ae bytes) 2024-09-21T15:34:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:5491 read 0x33606c7 thru 0x3363fb3 (0x38ed bytes) 2024-09-21T15:34:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:5492 writesame 0x1703129 thru 0x17093db data_size 0x1(0x62b3 bytes) 2024-09-21T15:34:47.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:47.663 INFO:teuthology.orchestra.run.smithi107.stdout:5493 punch from 0xb661754 to 0xb6683d5, (0x6c81 bytes) 2024-09-21T15:34:47.663 INFO:teuthology.orchestra.run.smithi107.stdout:5496 read 0xbc17e29 thru 0xbc23007 (0xb1df bytes) 2024-09-21T15:34:47.668 INFO:teuthology.orchestra.run.smithi107.stdout:5497 trunc from 0xe29ee32 to 0xa77b454 2024-09-21T15:34:47.668 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:47.663+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:47.678 INFO:teuthology.orchestra.run.smithi107.stdout:5498 read 0x2bced55 thru 0x2bdc70b (0xd9b7 bytes) 2024-09-21T15:34:47.690 INFO:teuthology.orchestra.run.smithi107.stdout:5499 trunc from 0xa77b454 to 0x1637942 2024-09-21T15:34:47.707 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-21T15:34:47.717 INFO:teuthology.orchestra.run.smithi107.stdout:5500 read 0x3f8697 thru 0x3f95e2 (0xf4c bytes) 2024-09-21T15:34:47.717 INFO:teuthology.orchestra.run.smithi107.stdout:5501 read 0xb9819d thru 0xba712f (0xef93 bytes) 2024-09-21T15:34:47.717 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.733 INFO:teuthology.orchestra.run.smithi107.stdout:5504 trunc from 0x1637942 to 0xed9589d 2024-09-21T15:34:47.743 INFO:teuthology.orchestra.run.smithi107.stdout:5505 trunc from 0xed9589d to 0xcc088de 2024-09-21T15:34:47.752 INFO:teuthology.orchestra.run.smithi107.stdout:5506 read 0x1906e66 thru 0x190f7b1 (0x894c bytes) 2024-09-21T15:34:47.753 INFO:teuthology.orchestra.run.smithi107.stdout:5508 punch from 0x9cbd739 to 0x9cbdb1b, (0x3e2 bytes) 2024-09-21T15:34:47.753 INFO:teuthology.orchestra.run.smithi107.stdout:5510 writesame 0x71ca146 thru 0x71ced0e data_size 0x3(0x4bc9 bytes) 2024-09-21T15:34:47.765 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.765 INFO:teuthology.orchestra.run.smithi107.stdout:5512 write 0xb8de2cc thru 0xb8ed107 (0xee3c bytes) 2024-09-21T15:34:47.769 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:47.767+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:47.770 INFO:teuthology.orchestra.run.smithi107.stdout:5514 read 0x46c1662 thru 0x46d00dc (0xea7b bytes) 2024-09-21T15:34:47.778 INFO:teuthology.orchestra.run.smithi107.stdout:5516 writesame 0xd0dd545 thru 0xd0e7f7d data_size 0x1(0xaa39 bytes) 2024-09-21T15:34:47.780 INFO:teuthology.orchestra.run.smithi107.stdout:5517 write 0x77b0e4e thru 0x77c07e1 (0xf994 bytes) 2024-09-21T15:34:47.786 INFO:teuthology.orchestra.run.smithi107.stdout:5518 punch from 0x3acb98f to 0x3ad25dc, (0x6c4d bytes) 2024-09-21T15:34:47.786 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.786 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.792 INFO:teuthology.orchestra.run.smithi107.stdout:5519 write 0xd8fa643 thru 0xd907c11 (0xd5cf bytes) 2024-09-21T15:34:47.795 INFO:teuthology.orchestra.run.smithi107.stdout:5520 punch from 0x86831d8 to 0x868b4f8, (0x8320 bytes) 2024-09-21T15:34:47.795 INFO:teuthology.orchestra.run.smithi107.stdout:5521 writesame 0x40a712f thru 0x40b2c91 data_size 0x4d(0xbb63 bytes) 2024-09-21T15:34:47.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:47.802 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.802 INFO:teuthology.orchestra.run.smithi107.stdout:5523 writesame 0x4511270 thru 0x451ee8d data_size 0x46(0xdc1e bytes) 2024-09-21T15:34:47.807 INFO:teuthology.orchestra.run.smithi107.stdout:5524 trunc from 0xd907c12 to 0xea49510 2024-09-21T15:34:47.815 INFO:teuthology.orchestra.run.smithi107.stdout:5525 read 0x580cbe2 thru 0x5813371 (0x6790 bytes) 2024-09-21T15:34:47.815 INFO:teuthology.orchestra.run.smithi107.stdout:5526 write 0xb598066 thru 0xb5a5011 (0xcfac bytes) 2024-09-21T15:34:47.818 INFO:teuthology.orchestra.run.smithi107.stdout:5527 read 0x95d0f9c thru 0x95d19ea (0xa4f bytes) 2024-09-21T15:34:47.818 INFO:teuthology.orchestra.run.smithi107.stdout:5528 write 0x609c731 thru 0x60a7fc3 (0xb893 bytes) 2024-09-21T15:34:47.824 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:47.824 INFO:teuthology.orchestra.run.smithi107.stdout:5531 punch from 0xb0c973a to 0xb0cf8ea, (0x61b0 bytes) 2024-09-21T15:34:47.840 INFO:teuthology.orchestra.run.smithi107.stdout:5532 trunc from 0xea49510 to 0x153da73 2024-09-21T15:34:47.870 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:47.867+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:47.877 INFO:teuthology.orchestra.run.smithi107.stdout:5535 trunc from 0x153da73 to 0x886e64e 2024-09-21T15:34:47.885 INFO:teuthology.orchestra.run.smithi107.stdout:5537 read 0x70561fa thru 0x7060c9f (0xaaa6 bytes) 2024-09-21T15:34:47.885 INFO:teuthology.orchestra.run.smithi107.stdout:5538 read 0x303378 thru 0x303719 (0x3a2 bytes) 2024-09-21T15:34:47.885 INFO:teuthology.orchestra.run.smithi107.stdout:5539 clone 41 order 22 su 2097152 sc 13 2024-09-21T15:34:47.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-21T15:34:47.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:34:47.933 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:47.934 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:47.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:47.992 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:47.990+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:48.071 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:48.070+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:48.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T15:34:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:48.102 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:48.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:48.207 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:48.206+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:48.273 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:48.271+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:48.277 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:48.277 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:48.289 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:48.374 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:48.374+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:48.421 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:48.422 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:48.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:48.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:48.481 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:48.479+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:48.556 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:48.557 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:48.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:48.574 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:48.574 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:48.575 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:48.570+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:48.589 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:48.589 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:48.604 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:48.604 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:48.620 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:48.620 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:48.636 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:48.653 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:48.654 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:48.670 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:48.670 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:48.676 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:48.674+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:48.689 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:48.689 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460404726, 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.740 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:34:48.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.744 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460380667000003, 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.745 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:34:48.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:34:48.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460356703999999, 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.749 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:34:48.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:34:48.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.752 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:48.753 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460332938000001, 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:34:48.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.758 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460309295999998, 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:34:48.762 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460283742999998, 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.763 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:34:48.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:34:48.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460239528000002, 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.767 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:34:48.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:34:48.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:34:48.770 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460210818999997, 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:34:48.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:34:48.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:34:48.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460179275999998, 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.775 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:34:48.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:34:48.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460152204000003, 2024-09-21T15:34:48.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:34:48.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:34:48.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.460126271, 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:34:48.783 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:34:48.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.459591328000002, 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.791 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.45896166, 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:34:48.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:34:48.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:48.796 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.455050096999997, 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:34:48.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.455003599000001, 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:34:48.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.454954430000001, 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:34:48.806 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:34:48.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:34:48.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:34:48.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.454906582, 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:34:48.810 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:34:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:34:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.454858193, 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.814 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.454777272000001, 2024-09-21T15:34:48.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:48.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:34:48.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.451178908999999, 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.823 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.825 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.826 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.827 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.827 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:48.827 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:48.827 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:48.828 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:48.828 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:48.826+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:34:48.929 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:48.927+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:48.957 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:48.957 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:48.957 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:48.957 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:48.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:48.983 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone40 from 0x886e64e (overlap 0x3bd90b) to 0x0 2024-09-21T15:34:49.030 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:49.026+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:49.092 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-21T15:34:49.131 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:49.126+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:49.137 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:49.137 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:49.137 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:49.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:49.233 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:49.231+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:49.286 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:49.286 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:49.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003870186999997, 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:34:49.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003811626000001, 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.291 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003740911000001, 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.295 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:49.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:34:49.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:34:49.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003691275999998, 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.299 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:34:49.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:49.303 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003623507999997, 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:49.307 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003512635, 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.311 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003418668000002, 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:49.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:34:49.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:34:49.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003370543000003, 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:49.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:34:49.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:34:49.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.319 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003300865, 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:34:49.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003251472999999, 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.325 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:34:49.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003204603999997, 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.329 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003157508999998, 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.333 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.335 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003108470999997, 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.337 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.003040722999998, 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.341 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.342 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:34:49.343 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.002992044000003, 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.344 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.345 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.346 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.347 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.002881760999998, 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.348 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.349 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.350 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:34:49.351 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.000633303999997, 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.352 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.353 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:34:49.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:34:49.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.000563552999999, 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:34:49.356 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:34:49.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:34:49.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.359 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:34:49.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:34:49.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.361 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.000473902000003, 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:34:49.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:34:49.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:34:49.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 42.000194120000003, 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.366 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.370 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:49.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:49.372 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:49.370+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:49.472 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:49.470+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:49.538 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:49.538 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:49.539 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:49.539 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:49.552 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:49.563 INFO:teuthology.orchestra.run.smithi107.stdout:5540 write 0x82853fe thru 0x828fdde (0xa9e1 bytes) 2024-09-21T15:34:49.573 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:49.570+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:49.587 INFO:teuthology.orchestra.run.smithi107.stdout:5541 trunc from 0x886e64e to 0x2287be5 2024-09-21T15:34:49.622 INFO:teuthology.orchestra.run.smithi107.stdout:5542 read 0x38fc0e thru 0x39101b (0x140e bytes) 2024-09-21T15:34:49.622 INFO:teuthology.orchestra.run.smithi107.stdout:5543 write 0xb38613 thru 0xb43081 (0xaa6f bytes) 2024-09-21T15:34:49.625 INFO:teuthology.orchestra.run.smithi107.stdout:5544 punch from 0x50e70c to 0x514c56, (0x654a bytes) 2024-09-21T15:34:49.638 INFO:teuthology.orchestra.run.smithi107.stdout:5545 write 0x5377599 thru 0x537fb38 (0x85a0 bytes) 2024-09-21T15:34:49.642 INFO:teuthology.orchestra.run.smithi107.stdout:5547 punch from 0x42ed6e1 to 0x42f28ec, (0x520b bytes) 2024-09-21T15:34:49.650 INFO:teuthology.orchestra.run.smithi107.stdout:5548 trunc from 0x537fb39 to 0x97fe6e4 2024-09-21T15:34:49.673 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:49.671+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:49.674 INFO:teuthology.orchestra.run.smithi107.stdout:5549 write 0xc1dcd72 thru 0xc1e808c (0xb31b bytes) 2024-09-21T15:34:49.676 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:49.676 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:49.676 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:49.676 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:49.676 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:49.680 INFO:teuthology.orchestra.run.smithi107.stdout:5550 writesame 0x8b90821 thru 0x8b9c899 data_size 0x1(0xc079 bytes) 2024-09-21T15:34:49.692 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:49.725 INFO:teuthology.orchestra.run.smithi107.stdout:5551 writesame 0x647c4d4 thru 0x648b046 data_size 0x5(0xeb73 bytes) 2024-09-21T15:34:49.745 INFO:teuthology.orchestra.run.smithi107.stdout:5552 write 0x889e670 thru 0x88a9056 (0xa9e7 bytes) 2024-09-21T15:34:49.752 INFO:teuthology.orchestra.run.smithi107.stdout:5556 punch from 0xb30e048 to 0xb31bcab, (0xdc63 bytes) 2024-09-21T15:34:49.752 INFO:teuthology.orchestra.run.smithi107.stdout:5557 punch from 0x95d0ca1 to 0x95d7a07, (0x6d66 bytes) 2024-09-21T15:34:49.752 INFO:teuthology.orchestra.run.smithi107.stdout:5560 read 0xb699fe8 thru 0xb69b406 (0x141f bytes) 2024-09-21T15:34:49.752 INFO:teuthology.orchestra.run.smithi107.stdout:5561 punch from 0x1d8b5d8 to 0x1d8b9e2, (0x40a bytes) 2024-09-21T15:34:49.752 INFO:teuthology.orchestra.run.smithi107.stdout:5562 writesame 0x9b8d45d thru 0x9b96123 data_size 0x3(0x8cc7 bytes) 2024-09-21T15:34:49.781 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:49.779+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:49.797 INFO:teuthology.orchestra.run.smithi107.stdout:5565 trunc from 0xc1e808d to 0x2810bc4 2024-09-21T15:34:49.821 INFO:teuthology.orchestra.run.smithi107.stdout:5566 read 0x87cb4f thru 0x888480 (0xb932 bytes) 2024-09-21T15:34:49.843 INFO:teuthology.orchestra.run.smithi107.stdout:5567 writesame 0x75fd837 thru 0x7603fbf data_size 0x5f(0x6789 bytes) 2024-09-21T15:34:49.865 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:49.865 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:49.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:49.865 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.204701, 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:34:49.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:34:49.869 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.200850418000002, 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.870 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.200825225999999, 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.196849204000003, 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.878 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.196823182000003, 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.882 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.196799628999997, 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.886 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.196764786999999, 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.890 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:34:49.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:34:49.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.192783265000003, 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:34:49.894 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.895 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:49.898 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.192675887999997, 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:49.902 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.192649318999997, 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:49.906 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.192621742999997, 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.908 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:49.910 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.192598232999998, 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188896606999997, 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188866853999997, 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:34:49.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.922 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188839784999999, 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.925 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.926 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188816449000001, 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:34:49.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:34:49.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:34:49.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188747845000002, 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.931 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.188719104999997, 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.935 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.184806121999998, 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.939 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 41.184751050000003, 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:49.943 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:49.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:49.947 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:5568 write 0xc6c0d17 thru 0xc6cf001 (0xe2eb bytes) 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:5569 trunc from 0xc6cf002 to 0x5e75b6f 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:5570 read 0xc0c3a8 thru 0xc0f188 (0x2de1 bytes) 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:5571 write 0xcf09bb thru 0xcf385c (0x2ea2 bytes) 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:49.949 INFO:teuthology.orchestra.run.smithi107.stdout:5575 writesame 0x73313d2 thru 0x733585d data_size 0x4(0x448c bytes) 2024-09-21T15:34:49.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:49.949 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:49.943+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:49.960 INFO:teuthology.orchestra.run.smithi107.stdout:5577 write 0xb1470d2 thru 0xb14daeb (0x6a1a bytes) 2024-09-21T15:34:49.972 INFO:teuthology.orchestra.run.smithi107.stdout:5579 trunc from 0xb14daec to 0xd003496 2024-09-21T15:34:49.986 INFO:teuthology.orchestra.run.smithi107.stdout:5580 write 0x70e75dc thru 0x70ee4eb (0x6f10 bytes) 2024-09-21T15:34:49.996 INFO:teuthology.orchestra.run.smithi107.stdout:5581 trunc from 0xd003496 to 0xc39475a 2024-09-21T15:34:50.012 INFO:teuthology.orchestra.run.smithi107.stdout:5582 read 0xa09631e thru 0xa09823a (0x1f1d bytes) 2024-09-21T15:34:50.012 INFO:teuthology.orchestra.run.smithi107.stdout:5583 punch from 0xb85206b to 0xb858895, (0x682a bytes) 2024-09-21T15:34:50.012 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.012 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.012 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.017 INFO:teuthology.orchestra.run.smithi107.stdout:5587 trunc from 0xc39475a to 0xe4af9dc 2024-09-21T15:34:50.030 INFO:teuthology.orchestra.run.smithi107.stdout:5588 read 0x1730210 thru 0x173fe2e (0xfc1f bytes) 2024-09-21T15:34:50.031 INFO:teuthology.orchestra.run.smithi107.stdout:5590 writesame 0x58bdde thru 0x5997d4 data_size 0x1(0xd9f7 bytes) 2024-09-21T15:34:50.049 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:50.047+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:50.079 INFO:teuthology.orchestra.run.smithi107.stdout:5593 write 0x35c2c71 thru 0x35d18c5 (0xec55 bytes) 2024-09-21T15:34:50.109 INFO:teuthology.orchestra.run.smithi107.stdout:5594 trunc from 0xe4af9dc to 0x5ab9add 2024-09-21T15:34:50.129 INFO:teuthology.orchestra.run.smithi107.stdout:5595 punch from 0x28c46cf to 0x28cf8fa, (0xb22b bytes) 2024-09-21T15:34:50.129 INFO:teuthology.orchestra.run.smithi107.stdout:5596 write 0x1a29405 thru 0x1a344cb (0xb0c7 bytes) 2024-09-21T15:34:50.135 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:50.135 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:50.135 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:50.135 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:50.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:50.151 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:50.147+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:50.175 INFO:teuthology.orchestra.run.smithi107.stdout:5598 write 0xae0f6b4 thru 0xae11825 (0x2172 bytes) 2024-09-21T15:34:50.180 INFO:teuthology.orchestra.run.smithi107.stdout:5600 read 0x1397584 thru 0x13a5569 (0xdfe6 bytes) 2024-09-21T15:34:50.181 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.196 INFO:teuthology.orchestra.run.smithi107.stdout:5603 write 0xc7c4947 thru 0xc7ce197 (0x9851 bytes) 2024-09-21T15:34:50.201 INFO:teuthology.orchestra.run.smithi107.stdout:5606 writesame 0xb0be9da thru 0xb0c3cb1 data_size 0x2c(0x52d8 bytes) 2024-09-21T15:34:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:5607 punch from 0xe22337 to 0xe2b5a0, (0x9269 bytes) 2024-09-21T15:34:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:5608 punch from 0x9616e2b to 0x962436c, (0xd541 bytes) 2024-09-21T15:34:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:5610 punch from 0x7a7a39b to 0x7a86e54, (0xcab9 bytes) 2024-09-21T15:34:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:5613 punch from 0x90ced67 to 0x90dc785, (0xda1e bytes) 2024-09-21T15:34:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:5614 write 0xaa44723 thru 0xaa4baed (0x73cb bytes) 2024-09-21T15:34:50.212 INFO:teuthology.orchestra.run.smithi107.stdout:5615 read 0x174ae53 thru 0x175292a (0x7ad8 bytes) 2024-09-21T15:34:50.227 INFO:teuthology.orchestra.run.smithi107.stdout:5616 write 0xdead4ea thru 0xdebc908 (0xf41f bytes) 2024-09-21T15:34:50.233 INFO:teuthology.orchestra.run.smithi107.stdout:5617 punch from 0x5c7045a to 0x5c7aeaa, (0xaa50 bytes) 2024-09-21T15:34:50.233 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.245 INFO:teuthology.orchestra.run.smithi107.stdout:5619 write 0xeae1040 thru 0xeae56f8 (0x46b9 bytes) 2024-09-21T15:34:50.250 INFO:teuthology.orchestra.run.smithi107.stdout:5620 read 0x25930d6 thru 0x25944c3 (0x13ee bytes) 2024-09-21T15:34:50.250 INFO:teuthology.orchestra.run.smithi107.stdout:5622 write 0x84ade0b thru 0x84bd440 (0xf636 bytes) 2024-09-21T15:34:50.251 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:50.247+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:50.255 INFO:teuthology.orchestra.run.smithi107.stdout:5624 write 0xd995872 thru 0xd99c13c (0x68cb bytes) 2024-09-21T15:34:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:5626 write 0x86ff9dc thru 0x870adda (0xb3ff bytes) 2024-09-21T15:34:50.266 INFO:teuthology.orchestra.run.smithi107.stdout:5627 write 0x93a09ee thru 0x93ad1f6 (0xc809 bytes) 2024-09-21T15:34:50.272 INFO:teuthology.orchestra.run.smithi107.stdout:5628 punch from 0x15894c3 to 0x158f778, (0x62b5 bytes) 2024-09-21T15:34:50.272 INFO:teuthology.orchestra.run.smithi107.stdout:5629 read 0xcb0f606 thru 0xcb16f0e (0x7909 bytes) 2024-09-21T15:34:50.272 INFO:teuthology.orchestra.run.smithi107.stdout:5630 write 0x3eb17b1 thru 0x3eb2784 (0xfd4 bytes) 2024-09-21T15:34:50.277 INFO:teuthology.orchestra.run.smithi107.stdout:5631 write 0x6ce812d thru 0x6cea9df (0x28b3 bytes) 2024-09-21T15:34:50.280 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.280 INFO:teuthology.orchestra.run.smithi107.stdout:5637 punch from 0x561a770 to 0x561e02f, (0x38bf bytes) 2024-09-21T15:34:50.280 INFO:teuthology.orchestra.run.smithi107.stdout:5640 read 0x43d2541 thru 0x43db718 (0x91d8 bytes) 2024-09-21T15:34:50.280 INFO:teuthology.orchestra.run.smithi107.stdout:5642 write 0x421c4c0 thru 0x421fd39 (0x387a bytes) 2024-09-21T15:34:50.287 INFO:teuthology.orchestra.run.smithi107.stdout:5643 write 0x6f5789 thru 0x6ff9ab (0xa223 bytes) 2024-09-21T15:34:50.293 INFO:teuthology.orchestra.run.smithi107.stdout:5644 punch from 0x948f8e to 0x94adae, (0x1e20 bytes) 2024-09-21T15:34:50.293 INFO:teuthology.orchestra.run.smithi107.stdout:5645 writesame 0x934103f thru 0x9342dd5 data_size 0x5(0x1d97 bytes) 2024-09-21T15:34:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:5646 read 0xc572046 thru 0xc5807ba (0xe775 bytes) 2024-09-21T15:34:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:5648 read 0x42a9820 thru 0x42acc95 (0x3476 bytes) 2024-09-21T15:34:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:5649 write 0xcbdb072 thru 0xcbe9cef (0xec7e bytes) 2024-09-21T15:34:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:5652 write 0x80f88c2 thru 0x80ff55e (0x6c9d bytes) 2024-09-21T15:34:50.306 INFO:teuthology.orchestra.run.smithi107.stdout:5654 read 0xe875d65 thru 0xe87f28b (0x9527 bytes) 2024-09-21T15:34:50.322 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:50.323 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:50.323 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:50.323 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:50.323 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:50.334 INFO:teuthology.orchestra.run.smithi107.stdout:5655 trunc from 0xeae56f9 to 0x2a105c4 2024-09-21T15:34:50.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:50.377 INFO:teuthology.orchestra.run.smithi107.stdout:5656 writesame 0x450f3e8 thru 0x451f0aa data_size 0x1(0xfcc3 bytes) 2024-09-21T15:34:50.385 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:50.383+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:50.395 INFO:teuthology.orchestra.run.smithi107.stdout:5657 write 0x7950600 thru 0x79545eb (0x3fec bytes) 2024-09-21T15:34:50.401 INFO:teuthology.orchestra.run.smithi107.stdout:5658 write 0x587812b thru 0x587bba6 (0x3a7c bytes) 2024-09-21T15:34:50.407 INFO:teuthology.orchestra.run.smithi107.stdout:5659 writesame 0x433a2c6 thru 0x434574d data_size 0x6d(0xb488 bytes) 2024-09-21T15:34:50.411 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.411 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.411 INFO:teuthology.orchestra.run.smithi107.stdout:5663 read 0x34c6bca thru 0x34c955e (0x2995 bytes) 2024-09-21T15:34:50.411 INFO:teuthology.orchestra.run.smithi107.stdout:5665 read 0x3e70945 thru 0x3e7148a (0xb46 bytes) 2024-09-21T15:34:50.412 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.422 INFO:teuthology.orchestra.run.smithi107.stdout:5667 trunc from 0x79545ec to 0x31c4fc7 2024-09-21T15:34:50.453 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:50.450+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:50.459 INFO:teuthology.orchestra.run.smithi107.stdout:5669 trunc from 0x31c4fc7 to 0xb378d19 2024-09-21T15:34:50.469 INFO:teuthology.orchestra.run.smithi107.stdout:5670 read 0x4c1d5eb thru 0x4c1f8fb (0x2311 bytes) 2024-09-21T15:34:50.470 INFO:teuthology.orchestra.run.smithi107.stdout:5672 writesame 0x5d92b49 thru 0x5d9379e data_size 0x1(0xc56 bytes) 2024-09-21T15:34:50.470 INFO:teuthology.orchestra.run.smithi107.stdout:5673 write 0x466a365 thru 0x4677c4b (0xd8e7 bytes) 2024-09-21T15:34:50.475 INFO:teuthology.orchestra.run.smithi107.stdout:5674 writesame 0x685c3c0 thru 0x6869bc4 data_size 0x1(0xd805 bytes) 2024-09-21T15:34:50.527 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:50.527 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.747500103999997, 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:50.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.530 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.531 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.747122113000003, 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.532 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:50.534 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.535 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:50.536 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.746902857999999, 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:50.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:50.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.746505695000003, 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:50.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.746101907000003, 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.545 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.745646145000002, 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.549 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:50.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:50.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.738419598999997, 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.553 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.738053311000002, 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:50.557 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.561 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.737341467999997, 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.565 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.734080921, 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.566 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:50.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:50.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.733640702000002, 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.570 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:50.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.572 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.574 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.733329279000003, 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:50.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:50.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.708401670999997, 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:50.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.583 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.707772015000003, 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:50.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.707367552000001, 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:50.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:50.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.706829159000002, 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:50.592 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.596 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.706302141000002, 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:50.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.705880415999999, 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:50.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:50.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.604 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.705465013999998, 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:50.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:50.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.608 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 55.704914586999998, 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.609 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:50.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:50.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:50.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:50.613 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:50.615 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:50.611+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:50.615 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:50.616 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.616 INFO:teuthology.orchestra.run.smithi107.stdout:5676 read 0x3e2a700 thru 0x3e3120a (0x6b0b bytes) 2024-09-21T15:34:50.616 INFO:teuthology.orchestra.run.smithi107.stdout:5677 write 0x76fb0ed thru 0x770799b (0xc8af bytes) 2024-09-21T15:34:50.616 INFO:teuthology.orchestra.run.smithi107.stdout:5678 punch from 0x30707c2 to 0x307fe6a, (0xf6a8 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5680 trunc from 0xb378d19 to 0x60645f9 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5681 read 0x26b5a7e thru 0x26b9be3 (0x4166 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5682 write 0xa08b81e thru 0xa095d70 (0xa553 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5683 writesame 0xd399d11 thru 0xd3a4208 data_size 0x4(0xa4f8 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5684 punch from 0x7cb34d2 to 0x7cb8998, (0x54c6 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5687 punch from 0xa617d59 to 0xa61d94b, (0x5bf2 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5688 punch from 0x8a3040 to 0x8a7531, (0x44f1 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5689 read 0x40dd2e4 thru 0x40eb61d (0xe33a bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5691 read 0x58ede9b thru 0x58edfc1 (0x127 bytes) 2024-09-21T15:34:50.617 INFO:teuthology.orchestra.run.smithi107.stdout:5692 read 0x7e4ae95 thru 0x7e519d3 (0x6b3f bytes) 2024-09-21T15:34:50.622 INFO:teuthology.orchestra.run.smithi107.stdout:5694 trunc from 0xd3a4209 to 0x7e7b87a 2024-09-21T15:34:50.654 INFO:teuthology.orchestra.run.smithi107.stdout:5695 write 0xb900d31 thru 0xb90348c (0x275c bytes) 2024-09-21T15:34:50.659 INFO:teuthology.orchestra.run.smithi107.stdout:5696 write 0x3bd2dcf thru 0x3bda604 (0x7836 bytes) 2024-09-21T15:34:50.665 INFO:teuthology.orchestra.run.smithi107.stdout:5697 write 0x3f911dd thru 0x3f94b4a (0x396e bytes) 2024-09-21T15:34:50.671 INFO:teuthology.orchestra.run.smithi107.stdout:5698 punch from 0x3a828e7 to 0x3a8e0ba, (0xb7d3 bytes) 2024-09-21T15:34:50.687 INFO:teuthology.orchestra.run.smithi107.stdout:5700 trunc from 0xb90348d to 0x21ef016 2024-09-21T15:34:50.705 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:50.715 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:50.711+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:50.722 INFO:teuthology.orchestra.run.smithi107.stdout:5703 writesame 0x6bd9e1e thru 0x6be1ff9 data_size 0x2(0x81dc bytes) 2024-09-21T15:34:50.722 INFO:teuthology.orchestra.run.smithi107.stdout:5705 read 0x48febce thru 0x4906c50 (0x8083 bytes) 2024-09-21T15:34:50.723 INFO:teuthology.orchestra.run.smithi107.stdout:5706 punch from 0x61e4f9 to 0x61f235, (0xd3c bytes) 2024-09-21T15:34:50.735 INFO:teuthology.orchestra.run.smithi107.stdout:5707 write 0x9e68d34 thru 0x9e75a59 (0xcd26 bytes) 2024-09-21T15:34:50.741 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.741 INFO:teuthology.orchestra.run.smithi107.stdout:5709 read 0x73b0af9 thru 0x73bd147 (0xc64f bytes) 2024-09-21T15:34:50.741 INFO:teuthology.orchestra.run.smithi107.stdout:5710 writesame 0x2c3b98c thru 0x2c454ec data_size 0x1(0x9b61 bytes) 2024-09-21T15:34:50.770 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:50.771 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:50.771 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:50.771 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:50.785 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:5711 punch from 0x164942b to 0x164a18d, (0xd62 bytes) 2024-09-21T15:34:50.796 INFO:teuthology.orchestra.run.smithi107.stdout:5713 writesame 0x1e20397 thru 0x1e28900 data_size 0x1(0x856a bytes) 2024-09-21T15:34:50.816 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:50.811+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:50.842 INFO:teuthology.orchestra.run.smithi107.stdout:5715 trunc from 0x9e75a5a to 0x5dcc6a 2024-09-21T15:34:50.851 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:50.851 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:50.870 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:50.870 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:50.897 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:34:50.904 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.905 INFO:teuthology.orchestra.run.smithi107.stdout:5720 read 0x414264 thru 0x423b51 (0xf8ee bytes) 2024-09-21T15:34:50.905 INFO:teuthology.orchestra.run.smithi107.stdout:5721 read 0x5d263b thru 0x5d56b7 (0x307d bytes) 2024-09-21T15:34:50.916 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:50.914+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:50.916 INFO:teuthology.orchestra.run.smithi107.stdout:5722 write 0x2bad065 thru 0x2baf816 (0x27b2 bytes) 2024-09-21T15:34:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.925 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:50.925 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:50.946 INFO:teuthology.orchestra.run.smithi107.stdout:5724 writesame 0xe258338 thru 0xe264b48 data_size 0x1(0xc811 bytes) 2024-09-21T15:34:50.946 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:50.947 INFO:teuthology.orchestra.run.smithi107.stdout:5726 writesame 0xb954d07 thru 0xb9640eb data_size 0x1(0xf3e5 bytes) 2024-09-21T15:34:50.949 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:50.983 INFO:teuthology.orchestra.run.smithi107.stdout:5729 write 0x30b2224 thru 0x30b5f7d (0x3d5a bytes) 2024-09-21T15:34:50.990 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.000 INFO:teuthology.orchestra.run.smithi107.stdout:5731 trunc from 0xe264b49 to 0x71c4050 2024-09-21T15:34:51.016 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:51.011+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:51.027 INFO:teuthology.orchestra.run.smithi107.stdout:5732 writesame 0x8e4b106 thru 0x8e5126f data_size 0x25(0x616a bytes) 2024-09-21T15:34:51.029 INFO:teuthology.orchestra.run.smithi107.stdout:5733 trunc from 0x8e51270 to 0x7f4e73f 2024-09-21T15:34:51.036 INFO:teuthology.orchestra.run.smithi107.stdout:5735 read 0x6db9a9b thru 0x6dc01d2 (0x6738 bytes) 2024-09-21T15:34:51.036 INFO:teuthology.orchestra.run.smithi107.stdout:5736 write 0x311e44 thru 0x320540 (0xe6fd bytes) 2024-09-21T15:34:51.050 INFO:teuthology.orchestra.run.smithi107.stdout:5737 writesame 0x8695697 thru 0x869f55f data_size 0x1(0x9ec9 bytes) 2024-09-21T15:34:51.050 INFO:teuthology.orchestra.run.smithi107.stdout:5738 writesame 0x68c21e3 thru 0x68cf78c data_size 0x7(0xd5aa bytes) 2024-09-21T15:34:51.065 INFO:teuthology.orchestra.run.smithi107.stdout:5739 read 0x3c329d6 thru 0x3c3aa70 (0x809b bytes) 2024-09-21T15:34:51.070 INFO:teuthology.orchestra.run.smithi107.stdout:5740 trunc from 0x869f560 to 0xb4be91f 2024-09-21T15:34:51.085 INFO:teuthology.orchestra.run.smithi107.stdout:5742 trunc from 0xb4be91f to 0x6f3392b 2024-09-21T15:34:51.090 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:51.090 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:51.095 INFO:teuthology.orchestra.run.smithi107.stdout:5743 punch from 0x1ea33a1 to 0x1eac16f, (0x8dce bytes) 2024-09-21T15:34:51.098 INFO:teuthology.orchestra.run.smithi107.stdout:5744 trunc from 0x6f3392b to 0x4f9ce4c 2024-09-21T15:34:51.117 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:51.114+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:51.119 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:51.119 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:51.123 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.123 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.125 INFO:teuthology.orchestra.run.smithi107.stdout:5747 write 0x6e0ce16 thru 0x6e1bb1d (0xed08 bytes) 2024-09-21T15:34:51.130 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.130 INFO:teuthology.orchestra.run.smithi107.stdout:5751 read 0x4600f38 thru 0x46085f6 (0x76bf bytes) 2024-09-21T15:34:51.130 INFO:teuthology.orchestra.run.smithi107.stdout:5752 write 0x44040a6 thru 0x440c6e7 (0x8642 bytes) 2024-09-21T15:34:51.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:51.142 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:51.142 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:51.143 INFO:teuthology.orchestra.run.smithi107.stdout:5754 write 0x6e6655e thru 0x6e72451 (0xbef4 bytes) 2024-09-21T15:34:51.153 INFO:teuthology.orchestra.run.smithi107.stdout:5755 trunc from 0x6e72452 to 0xcb165a5 2024-09-21T15:34:51.161 INFO:teuthology.orchestra.run.smithi107.stdout:5757 punch from 0x3659d8f to 0x3665eb5, (0xc126 bytes) 2024-09-21T15:34:51.161 INFO:teuthology.orchestra.run.smithi107.stdout:5759 write 0x10630f4 thru 0x106a5f4 (0x7501 bytes) 2024-09-21T15:34:51.164 INFO:teuthology.orchestra.run.smithi107.stdout:5762 read 0x3148783 thru 0x315605f (0xd8dd bytes) 2024-09-21T15:34:51.165 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:51.165 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:51.172 INFO:teuthology.orchestra.run.smithi107.stdout:5763 trunc from 0xcb165a5 to 0x75143c1 2024-09-21T15:34:51.190 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:51.190 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:51.191 INFO:teuthology.orchestra.run.smithi107.stdout:5766 write 0x7ce53cc thru 0x7ce7453 (0x2088 bytes) 2024-09-21T15:34:51.198 INFO:teuthology.orchestra.run.smithi107.stdout:5767 punch from 0x2ec19ba to 0x2ec9d7a, (0x83c0 bytes) 2024-09-21T15:34:51.198 INFO:teuthology.orchestra.run.smithi107.stdout:5768 writesame 0x141d99 thru 0x14c7bb data_size 0x1f(0xaa23 bytes) 2024-09-21T15:34:51.207 INFO:teuthology.orchestra.run.smithi107.stdout:5769 trunc from 0x7ce7454 to 0x9caa786 2024-09-21T15:34:51.211 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:51.211 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:51.215 INFO:teuthology.orchestra.run.smithi107.stdout:5770 writesame 0x3bbf5b1 thru 0x3bbfb6b data_size 0x3(0x5bb bytes) 2024-09-21T15:34:51.218 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:51.214+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:51.237 INFO:teuthology.orchestra.run.smithi107.stdout:5774 trunc from 0x9caa786 to 0x37e634 2024-09-21T15:34:51.271 INFO:teuthology.orchestra.run.smithi107.stdout:5775 punch from 0x28268a to 0x2888cc, (0x6242 bytes) 2024-09-21T15:34:51.271 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.271 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.281 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.281 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.281 INFO:teuthology.orchestra.run.smithi107.stdout:5779 trunc from 0x37e634 to 0x7d1e4e9 2024-09-21T15:34:51.289 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.299 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:51.306 INFO:teuthology.orchestra.run.smithi107.stdout:5781 write 0xe5c7240 thru 0xe5cfe4a (0x8c0b bytes) 2024-09-21T15:34:51.313 INFO:teuthology.orchestra.run.smithi107.stdout:5782 writesame 0x79ba851 thru 0x79c9a89 data_size 0x1(0xf239 bytes) 2024-09-21T15:34:51.319 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:51.315+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:51.382 INFO:teuthology.orchestra.run.smithi107.stdout:5783 read 0x52320a7 thru 0x5234f3d (0x2e97 bytes) 2024-09-21T15:34:51.396 INFO:teuthology.orchestra.run.smithi107.stdout:5784 trunc from 0xe5cfe4b to 0x73de1e7 2024-09-21T15:34:51.419 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:51.415+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:51.428 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.428 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.437 INFO:teuthology.orchestra.run.smithi107.stdout:5785 writesame 0x93e620f thru 0x93eeb5f data_size 0x1(0x8951 bytes) 2024-09-21T15:34:51.437 INFO:teuthology.orchestra.run.smithi107.stdout:5786 read 0x243d06f thru 0x244bc75 (0xec07 bytes) 2024-09-21T15:34:51.437 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:51.448 INFO:teuthology.orchestra.run.smithi107.stdout:5788 write 0x98ed355 thru 0x98f849b (0xb147 bytes) 2024-09-21T15:34:51.455 INFO:teuthology.orchestra.run.smithi107.stdout:5789 write 0x79ed45e thru 0x79fa5a5 (0xd148 bytes) 2024-09-21T15:34:51.476 INFO:teuthology.orchestra.run.smithi107.stdout:5790 writesame 0xea52f93 thru 0xea62937 data_size 0x51(0xf9a5 bytes) 2024-09-21T15:34:51.476 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.476 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.477 INFO:teuthology.orchestra.run.smithi107.stdout:5793 write 0x32bcad8 thru 0x32cad8c (0xe2b5 bytes) 2024-09-21T15:34:51.481 INFO:teuthology.orchestra.run.smithi107.stdout:5794 read 0x9b1944b thru 0x9b1d495 (0x404b bytes) 2024-09-21T15:34:51.481 INFO:teuthology.orchestra.run.smithi107.stdout:5795 punch from 0x39b6f40 to 0x39c5449, (0xe509 bytes) 2024-09-21T15:34:51.481 INFO:teuthology.orchestra.run.smithi107.stdout:5796 writesame 0xa335af2 thru 0xa344fd8 data_size 0x1(0xf4e7 bytes) 2024-09-21T15:34:51.521 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:51.519+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:51.544 INFO:teuthology.orchestra.run.smithi107.stdout:5797 trunc from 0xea62938 to 0x38bb80 2024-09-21T15:34:51.584 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.584 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.589 INFO:teuthology.orchestra.run.smithi107.stdout:5800 writesame 0x771c7e6 thru 0x771eccb data_size 0x1(0x24e6 bytes) 2024-09-21T15:34:51.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5801 writesame 0xdee8ed1 thru 0xdef3964 data_size 0x12(0xaa94 bytes) 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5803 read 0x4e2f11e thru 0x4e3b745 (0xc628 bytes) 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5805 read 0x8c65685 thru 0x8c74715 (0xf091 bytes) 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5807 punch from 0x10b77c7 to 0x10c442a, (0xcc63 bytes) 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5808 read 0x1911445 thru 0x191b3e5 (0x9fa1 bytes) 2024-09-21T15:34:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:5810 writesame 0xf80011 thru 0xf803ca data_size 0x9(0x3ba bytes) 2024-09-21T15:34:51.613 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.620 INFO:teuthology.orchestra.run.smithi107.stdout:5815 trunc from 0xdef3965 to 0x9bcec3e 2024-09-21T15:34:51.621 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:51.619+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:51.628 INFO:teuthology.orchestra.run.smithi107.stdout:5816 write 0x656d576 thru 0x6572ccb (0x5756 bytes) 2024-09-21T15:34:51.633 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.633 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.633 INFO:teuthology.orchestra.run.smithi107.stdout:5821 read 0x303a5e0 thru 0x303b4f3 (0xf14 bytes) 2024-09-21T15:34:51.633 INFO:teuthology.orchestra.run.smithi107.stdout:5823 punch from 0x432b7ae to 0x4332209, (0x6a5b bytes) 2024-09-21T15:34:51.633 INFO:teuthology.orchestra.run.smithi107.stdout:5824 write 0x615a3f1 thru 0x6166bba (0xc7ca bytes) 2024-09-21T15:34:51.639 INFO:teuthology.orchestra.run.smithi107.stdout:5825 punch from 0x2c5116c to 0x2c560e4, (0x4f78 bytes) 2024-09-21T15:34:51.639 INFO:teuthology.orchestra.run.smithi107.stdout:5826 read 0x10f6c14 thru 0x1104bb0 (0xdf9d bytes) 2024-09-21T15:34:51.648 INFO:teuthology.orchestra.run.smithi107.stdout:5827 trunc from 0x9bcec3e to 0x48f4243 2024-09-21T15:34:51.663 INFO:teuthology.orchestra.run.smithi107.stdout:5829 read 0x3728086 thru 0x37369b6 (0xe931 bytes) 2024-09-21T15:34:51.664 INFO:teuthology.orchestra.run.smithi107.stdout:5830 trunc from 0x48f4243 to 0x56363bf 2024-09-21T15:34:51.671 INFO:teuthology.orchestra.run.smithi107.stdout:5831 read 0x36d87a6 thru 0x36e7b13 (0xf36e bytes) 2024-09-21T15:34:51.678 INFO:teuthology.orchestra.run.smithi107.stdout:5832 trunc from 0x56363bf to 0x2749fec 2024-09-21T15:34:51.700 INFO:teuthology.orchestra.run.smithi107.stdout:5833 writesame 0xd2996e thru 0xd38994 data_size 0xcf(0xf027 bytes) 2024-09-21T15:34:51.707 INFO:teuthology.orchestra.run.smithi107.stdout:5834 punch from 0xd148cc to 0xd18836, (0x3f6a bytes) 2024-09-21T15:34:51.707 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.707 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.707 INFO:teuthology.orchestra.run.smithi107.stdout:5838 read 0x2558338 thru 0x256595a (0xd623 bytes) 2024-09-21T15:34:51.714 INFO:teuthology.orchestra.run.smithi107.stdout:5839 trunc from 0x2749fec to 0x5d7038e 2024-09-21T15:34:51.722 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:51.718+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:51.725 INFO:teuthology.orchestra.run.smithi107.stdout:5843 writesame 0x25b56ce thru 0x25c1d2e data_size 0x7(0xc661 bytes) 2024-09-21T15:34:51.739 INFO:teuthology.orchestra.run.smithi107.stdout:5844 read 0x2c4f579 thru 0x2c5d7e8 (0xe270 bytes) 2024-09-21T15:34:51.740 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.740 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.749 INFO:teuthology.orchestra.run.smithi107.stdout:5845 write 0x5e6eb3a thru 0x5e79cf5 (0xb1bc bytes) 2024-09-21T15:34:51.754 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:51.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:51.769 INFO:teuthology.orchestra.run.smithi107.stdout:5848 write 0x85c331d thru 0x85d03fc (0xd0e0 bytes) 2024-09-21T15:34:51.790 INFO:teuthology.orchestra.run.smithi107.stdout:5849 write 0xaa53973 thru 0xaa5ee6c (0xb4fa bytes) 2024-09-21T15:34:51.796 INFO:teuthology.orchestra.run.smithi107.stdout:5850 write 0x5f45f32 thru 0x5f4d84a (0x7919 bytes) 2024-09-21T15:34:51.814 INFO:teuthology.orchestra.run.smithi107.stdout:5851 trunc from 0xaa5ee6d to 0x34b1aa 2024-09-21T15:34:51.823 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:51.818+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:51.849 INFO:teuthology.orchestra.run.smithi107.stdout:5853 read 0x66df8 thru 0x74f3a (0xe143 bytes) 2024-09-21T15:34:51.850 INFO:teuthology.orchestra.run.smithi107.stdout:5854 read 0x380fa thru 0x453b0 (0xd2b7 bytes) 2024-09-21T15:34:51.897 INFO:teuthology.orchestra.run.smithi107.stdout:5855 writesame 0xed7bc5a thru 0xed8917e data_size 0x7(0xd525 bytes) 2024-09-21T15:34:51.897 INFO:teuthology.orchestra.run.smithi107.stdout:5856 write 0x2955f04 thru 0x2964a27 (0xeb24 bytes) 2024-09-21T15:34:51.900 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:34:51.901 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:51.901 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:51.913 INFO:teuthology.orchestra.run.smithi107.stdout:5857 punch from 0x5b3047f to 0x5b36272, (0x5df3 bytes) 2024-09-21T15:34:51.913 INFO:teuthology.orchestra.run.smithi107.stdout:5858 trunc from 0xed8917f to 0xe9afc74 2024-09-21T15:34:51.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:51.923 INFO:teuthology.orchestra.run.smithi107.stdout:5859 write 0x2101e48 thru 0x2109c99 (0x7e52 bytes) 2024-09-21T15:34:51.923 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:51.918+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:51.929 INFO:teuthology.orchestra.run.smithi107.stdout:5860 punch from 0xb080577 to 0xb08a72a, (0xa1b3 bytes) 2024-09-21T15:34:51.937 INFO:teuthology.orchestra.run.smithi107.stdout:5861 trunc from 0xe9afc74 to 0x9abac1f 2024-09-21T15:34:51.959 INFO:teuthology.orchestra.run.smithi107.stdout:5862 write 0xe1ace44 thru 0xe1bba98 (0xec55 bytes) 2024-09-21T15:34:51.964 INFO:teuthology.orchestra.run.smithi107.stdout:5863 read 0x8bcf137 thru 0x8bdd943 (0xe80d bytes) 2024-09-21T15:34:51.964 INFO:teuthology.orchestra.run.smithi107.stdout:5864 read 0x69b00cb thru 0x69bd74c (0xd682 bytes) 2024-09-21T15:34:51.964 INFO:teuthology.orchestra.run.smithi107.stdout:5865 write 0xd67637a thru 0xd680804 (0xa48b bytes) 2024-09-21T15:34:51.970 INFO:teuthology.orchestra.run.smithi107.stdout:5866 read 0x91373e8 thru 0x9139142 (0x1d5b bytes) 2024-09-21T15:34:51.970 INFO:teuthology.orchestra.run.smithi107.stdout:5867 read 0xa96b177 thru 0xa972bf4 (0x7a7e bytes) 2024-09-21T15:34:51.970 INFO:teuthology.orchestra.run.smithi107.stdout:5868 read 0xf395d1 thru 0xf400d1 (0x6b01 bytes) 2024-09-21T15:34:51.977 INFO:teuthology.orchestra.run.smithi107.stdout:5869 trunc from 0xe1bba99 to 0x95dfd13 2024-09-21T15:34:51.994 INFO:teuthology.orchestra.run.smithi107.stdout:5870 read 0x161341a thru 0x161836f (0x4f56 bytes) 2024-09-21T15:34:51.994 INFO:teuthology.orchestra.run.smithi107.stdout:5872 write 0x958b9c5 thru 0x958d087 (0x16c3 bytes) 2024-09-21T15:34:52.002 INFO:teuthology.orchestra.run.smithi107.stdout:5874 read 0x638d87e thru 0x638fea8 (0x262b bytes) 2024-09-21T15:34:52.002 INFO:teuthology.orchestra.run.smithi107.stdout:5876 writesame 0x31b2d0 thru 0x32124f data_size 0x40(0x5f80 bytes) 2024-09-21T15:34:52.006 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.018 INFO:teuthology.orchestra.run.smithi107.stdout:5880 write 0xbac967b thru 0xbacf362 (0x5ce8 bytes) 2024-09-21T15:34:52.024 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.022+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.024 INFO:teuthology.orchestra.run.smithi107.stdout:5882 write 0x9f97e95 thru 0x9f9bf30 (0x409c bytes) 2024-09-21T15:34:52.030 INFO:teuthology.orchestra.run.smithi107.stdout:5885 write 0x1e88cea thru 0x1e91291 (0x85a8 bytes) 2024-09-21T15:34:52.037 INFO:teuthology.orchestra.run.smithi107.stdout:5886 read 0x20abfa3 thru 0x20b43c3 (0x8421 bytes) 2024-09-21T15:34:52.038 INFO:teuthology.orchestra.run.smithi107.stdout:5887 read 0x221fb24 thru 0x22220d2 (0x25af bytes) 2024-09-21T15:34:52.038 INFO:teuthology.orchestra.run.smithi107.stdout:5888 read 0x1829c24 thru 0x1832aa2 (0x8e7f bytes) 2024-09-21T15:34:52.038 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.039 INFO:teuthology.orchestra.run.smithi107.stdout:5890 read 0x8c16129 thru 0x8c1c774 (0x664c bytes) 2024-09-21T15:34:52.046 INFO:teuthology.orchestra.run.smithi107.stdout:5891 writesame 0xc269b95 thru 0xc26ea44 data_size 0x8(0x4eb0 bytes) 2024-09-21T15:34:52.046 INFO:teuthology.orchestra.run.smithi107.stdout:5892 write 0x7359786 thru 0x73662be (0xcb39 bytes) 2024-09-21T15:34:52.046 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.047 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.052 INFO:teuthology.orchestra.run.smithi107.stdout:5893 read 0x8f7719e thru 0x8f82b5a (0xb9bd bytes) 2024-09-21T15:34:52.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:52.067 INFO:teuthology.orchestra.run.smithi107.stdout:5894 trunc from 0xc26ea45 to 0x27a2728 2024-09-21T15:34:52.101 INFO:teuthology.orchestra.run.smithi107.stdout:5895 writesame 0x7086890 thru 0x708a029 data_size 0xb(0x379a bytes) 2024-09-21T15:34:52.101 INFO:teuthology.orchestra.run.smithi107.stdout:5897 write 0x415b6fd thru 0x41630eb (0x79ef bytes) 2024-09-21T15:34:52.107 INFO:teuthology.orchestra.run.smithi107.stdout:5900 read 0x282e8c6 thru 0x283be40 (0xd57b bytes) 2024-09-21T15:34:52.108 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.108 INFO:teuthology.orchestra.run.smithi107.stdout:5904 punch from 0x5ec1aa5 to 0x5ed0d90, (0xf2eb bytes) 2024-09-21T15:34:52.108 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.108 INFO:teuthology.orchestra.run.smithi107.stdout:5906 punch from 0x43c945b to 0x43c946c, (0x11 bytes) 2024-09-21T15:34:52.119 INFO:teuthology.orchestra.run.smithi107.stdout:5907 trunc from 0x708a02a to 0xe8e84ef 2024-09-21T15:34:52.126 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:52.123+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:52.126 INFO:teuthology.orchestra.run.smithi107.stdout:5908 write 0x5dd17f thru 0x5eb892 (0xe714 bytes) 2024-09-21T15:34:52.130 INFO:teuthology.orchestra.run.smithi107.stdout:5909 read 0x6b5d5a3 thru 0x6b62e65 (0x58c3 bytes) 2024-09-21T15:34:52.130 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.130 INFO:teuthology.orchestra.run.smithi107.stdout:5911 read 0xdb16060 thru 0xdb1e405 (0x83a6 bytes) 2024-09-21T15:34:52.130 INFO:teuthology.orchestra.run.smithi107.stdout:5912 read 0x7ac2c7c thru 0x7ad228f (0xf614 bytes) 2024-09-21T15:34:52.130 INFO:teuthology.orchestra.run.smithi107.stdout:5913 read 0x2033760 thru 0x203c44b (0x8cec bytes) 2024-09-21T15:34:52.131 INFO:teuthology.orchestra.run.smithi107.stdout:5914 write 0xaef4681 thru 0xaef8df9 (0x4779 bytes) 2024-09-21T15:34:52.135 INFO:teuthology.orchestra.run.smithi107.stdout:5915 write 0x7b04d2c thru 0x7b0c0ce (0x73a3 bytes) 2024-09-21T15:34:52.157 INFO:teuthology.orchestra.run.smithi107.stdout:5916 trunc from 0xe8e84ef to 0x62084a0 2024-09-21T15:34:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.186 INFO:teuthology.orchestra.run.smithi107.stdout:5918 write 0x6aff15b thru 0x6b02535 (0x33db bytes) 2024-09-21T15:34:52.194 INFO:teuthology.orchestra.run.smithi107.stdout:5919 write 0xff6b77 thru 0x1004a33 (0xdebd bytes) 2024-09-21T15:34:52.198 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.198 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:52.221 INFO:teuthology.orchestra.run.smithi107.stdout:5921 writesame 0xa1307fd thru 0xa13dec5 data_size 0x7(0xd6c9 bytes) 2024-09-21T15:34:52.232 INFO:teuthology.orchestra.run.smithi107.stdout:5922 trunc from 0xa13dec6 to 0x52f1e25 2024-09-21T15:34:52.233 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.230+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.276 INFO:teuthology.orchestra.run.smithi107.stdout:5923 write 0xdca75f6 thru 0xdcb5aa7 (0xe4b2 bytes) 2024-09-21T15:34:52.283 INFO:teuthology.orchestra.run.smithi107.stdout:5924 writesame 0x79db57d thru 0x79e62e9 data_size 0x3(0xad6d bytes) 2024-09-21T15:34:52.309 INFO:teuthology.orchestra.run.smithi107.stdout:5925 write 0xa519a70 thru 0xa51f1d1 (0x5762 bytes) 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:5927 read 0x9b70a4b thru 0x9b782fe (0x78b4 bytes) 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:5928 punch from 0x9d061ac to 0x9d106f6, (0xa54a bytes) 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:5931 punch from 0x2725b06 to 0x2734d6b, (0xf265 bytes) 2024-09-21T15:34:52.316 INFO:teuthology.orchestra.run.smithi107.stdout:5932 write 0x9a9c5f5 thru 0x9aab605 (0xf011 bytes) 2024-09-21T15:34:52.323 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.323 INFO:teuthology.orchestra.run.smithi107.stdout:5935 write 0x74f684a thru 0x74f7809 (0xfc0 bytes) 2024-09-21T15:34:52.327 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:52.323+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:52.329 INFO:teuthology.orchestra.run.smithi107.stdout:5936 read 0x5bdd82c thru 0x5bea2f7 (0xcacc bytes) 2024-09-21T15:34:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:5937 punch from 0x90d774f to 0x90d7f1b, (0x7cc bytes) 2024-09-21T15:34:52.330 INFO:teuthology.orchestra.run.smithi107.stdout:5939 writesame 0x1bbf5b3 thru 0x1bccd04 data_size 0x3(0xd752 bytes) 2024-09-21T15:34:52.350 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.350 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.357 INFO:teuthology.orchestra.run.smithi107.stdout:5940 punch from 0x7a3fc40 to 0x7a4049c, (0x85c bytes) 2024-09-21T15:34:52.357 INFO:teuthology.orchestra.run.smithi107.stdout:5941 punch from 0x2e73d5b to 0x2e7f908, (0xbbad bytes) 2024-09-21T15:34:52.367 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:52.380 INFO:teuthology.orchestra.run.smithi107.stdout:5942 trunc from 0xdcb5aa8 to 0x353b6f0 2024-09-21T15:34:52.421 INFO:teuthology.orchestra.run.smithi107.stdout:5943 write 0x3b2adf0 thru 0x3b2b0f4 (0x305 bytes) 2024-09-21T15:34:52.428 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.428 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:52.423+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:52.433 INFO:teuthology.orchestra.run.smithi107.stdout:5945 trunc from 0x3b2b0f5 to 0x946dd3 2024-09-21T15:34:52.477 INFO:teuthology.orchestra.run.smithi107.stdout:5946 trunc from 0x946dd3 to 0x4b1cf26 2024-09-21T15:34:52.487 INFO:teuthology.orchestra.run.smithi107.stdout:5950 punch from 0x3d9024a to 0x3d9aeff, (0xacb5 bytes) 2024-09-21T15:34:52.490 INFO:teuthology.orchestra.run.smithi107.stdout:5952 trunc from 0x4b1cf26 to 0x64e8599 2024-09-21T15:34:52.503 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.503 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:52.526 INFO:teuthology.orchestra.run.smithi107.stdout:5953 writesame 0xbae2e3e thru 0xbaee45a data_size 0x1(0xb61d bytes) 2024-09-21T15:34:52.526 INFO:teuthology.orchestra.run.smithi107.stdout:5954 read 0x80c5ded thru 0x80cdfb7 (0x81cb bytes) 2024-09-21T15:34:52.526 INFO:teuthology.orchestra.run.smithi107.stdout:5956 read 0x2375847 thru 0x237a45b (0x4c15 bytes) 2024-09-21T15:34:52.526 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.529 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.526+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.539 INFO:teuthology.orchestra.run.smithi107.stdout:5959 writesame 0xd2a140c thru 0xd2b08c7 data_size 0x8a(0xf4bc bytes) 2024-09-21T15:34:52.539 INFO:teuthology.orchestra.run.smithi107.stdout:5961 write 0xb695ff2 thru 0xb69b61f (0x562e bytes) 2024-09-21T15:34:52.545 INFO:teuthology.orchestra.run.smithi107.stdout:5962 punch from 0x5f95aa3 to 0x5fa4ff0, (0xf54d bytes) 2024-09-21T15:34:52.545 INFO:teuthology.orchestra.run.smithi107.stdout:5963 punch from 0x86d8213 to 0x86e5d87, (0xdb74 bytes) 2024-09-21T15:34:52.545 INFO:teuthology.orchestra.run.smithi107.stdout:5965 write 0x7ef0872 thru 0x7effac8 (0xf257 bytes) 2024-09-21T15:34:52.552 INFO:teuthology.orchestra.run.smithi107.stdout:5966 write 0x46e3920 thru 0x46e8068 (0x4749 bytes) 2024-09-21T15:34:52.557 INFO:teuthology.orchestra.run.smithi107.stdout:5968 write 0x30abad3 thru 0x30b15e4 (0x5b12 bytes) 2024-09-21T15:34:52.563 INFO:teuthology.orchestra.run.smithi107.stdout:5969 write 0x323ee7a thru 0x324bf04 (0xd08b bytes) 2024-09-21T15:34:52.569 INFO:teuthology.orchestra.run.smithi107.stdout:5970 read 0x5ea105d thru 0x5eaed30 (0xdcd4 bytes) 2024-09-21T15:34:52.569 INFO:teuthology.orchestra.run.smithi107.stdout:5971 writesame 0x315adcb thru 0x31624b8 data_size 0xd(0x76ee bytes) 2024-09-21T15:34:52.572 INFO:teuthology.orchestra.run.smithi107.stdout:5972 read 0x8f4c013 thru 0x8f55f81 (0x9f6f bytes) 2024-09-21T15:34:52.572 INFO:teuthology.orchestra.run.smithi107.stdout:5973 write 0x4d693fe thru 0x4d77615 (0xe218 bytes) 2024-09-21T15:34:52.578 INFO:teuthology.orchestra.run.smithi107.stdout:5974 writesame 0x104a4f8 thru 0x104f1b1 data_size 0x7(0x4cba bytes) 2024-09-21T15:34:52.586 INFO:teuthology.orchestra.run.smithi107.stdout:5975 writesame 0xbbb5c95 thru 0xbbbc352 data_size 0x1(0x66be bytes) 2024-09-21T15:34:52.615 INFO:teuthology.orchestra.run.smithi107.stdout:5976 punch from 0xb77cd3 to 0xb7b48c, (0x37b9 bytes) 2024-09-21T15:34:52.630 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.626+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.631 INFO:teuthology.orchestra.run.smithi107.stdout:5978 trunc from 0xd2b08c8 to 0x6bd175f 2024-09-21T15:34:52.660 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.660 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:52.681 INFO:teuthology.orchestra.run.smithi107.stdout:5979 write 0x8279832 thru 0x827cc5f (0x342e bytes) 2024-09-21T15:34:52.687 INFO:teuthology.orchestra.run.smithi107.stdout:5980 read 0xab1338 thru 0xab3fcb (0x2c94 bytes) 2024-09-21T15:34:52.687 INFO:teuthology.orchestra.run.smithi107.stdout:5981 read 0x2ec0ed2 thru 0x2ec3daf (0x2ede bytes) 2024-09-21T15:34:52.687 INFO:teuthology.orchestra.run.smithi107.stdout:compare and write disabled 2024-09-21T15:34:52.687 INFO:teuthology.orchestra.run.smithi107.stdout:5983 writesame 0x17e4669 thru 0x17f3db1 data_size 0xb(0xf749 bytes) 2024-09-21T15:34:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:5986 punch from 0x1b3e9d8 to 0x1b40276, (0x189e bytes) 2024-09-21T15:34:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:5988 read 0x52c7eda thru 0x52d5c06 (0xdd2d bytes) 2024-09-21T15:34:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:5989 read 0x67e4729 thru 0x67e6390 (0x1c68 bytes) 2024-09-21T15:34:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:5990 read 0x2bdc51e thru 0x2beb34c (0xee2f bytes) 2024-09-21T15:34:52.693 INFO:teuthology.orchestra.run.smithi107.stdout:5991 read 0x3182e44 thru 0x318e42e (0xb5eb bytes) 2024-09-21T15:34:52.698 INFO:teuthology.orchestra.run.smithi107.stdout:5992 trunc from 0x827cc60 to 0x6da145c 2024-09-21T15:34:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:5993 punch from 0x1fb7d21 to 0x1fbf271, (0x7550 bytes) 2024-09-21T15:34:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:5994 writesame 0x12be296 thru 0x12c246d data_size 0x56(0x41d8 bytes) 2024-09-21T15:34:52.720 INFO:teuthology.orchestra.run.smithi107.stdout:5995 punch from 0x348927e to 0x3491565, (0x82e7 bytes) 2024-09-21T15:34:52.731 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.726+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.743 INFO:teuthology.orchestra.run.smithi107.stdout:6000 write 0xade5b8d thru 0xade9993 (0x3e07 bytes) 2024-09-21T15:34:52.781 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-21T15:34:52.800 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.800 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:52.861 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:52.859+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:52.932 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:52.930+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:52.934 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:52.934 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:52.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:53.033 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:53.030+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:53.116 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:53.117 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:53.130 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:53.134 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:53.130+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:53.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:53.234 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:53.231+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:53.250 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:53.251 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:53.255 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:34:53.255 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:34:53.255 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:34:53.255 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:34:53.265 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:53.357 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:53.355+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:53.388 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:53.388 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:53.405 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:53.405 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:53.421 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:53.422 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:53.433 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:53.433 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:53.436 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:53.431+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:53.437 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:53.447 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:53.455 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:53.474 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:53.494 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:53.494 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:53.511 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:53.511 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:53.537 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:53.535+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 111, 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:34:51.881438+0000", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:33:20.030364+0000", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5804032, 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:34:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "nodeep-scrub", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:34:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "luminous", 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:34:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:34:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:34:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:34:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:34:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:34:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:34:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "110", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:34:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 44, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 110, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:34:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:34:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:34:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 101, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:34:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 97, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 89, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 100, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:34:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 88, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:34:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 47, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 98, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:34:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:34:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 101, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:34:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:34:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:34:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:34:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:34:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [ 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.10", 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: 7, 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: 5 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.14", 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: 2, 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: 6 2024-09-21T15:34:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:34:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [], 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [], 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:53.647 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:53.649 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:53.649 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:53.649 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:53.650 INFO:tasks.thrashosds.thrasher:j is {'epoch': 111, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:34:51.881438+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:33:20.030364+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.10', 'osds': [7, 5]}, {'pgid': '3.14', 'osds': [2, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:34:53.650 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.10 2024-09-21T15:34:53.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.10 2024-09-21T15:34:53.651 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:53.651 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:53.647+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:53.750 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:53.746+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:53.817 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:53.818 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:53.818 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:53.818 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:53.818 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:53.831 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:53.851 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:53.846+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:53.925 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:34:53.957 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:53.957 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.298140439000001, 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.958 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:34:53.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.294289857000003, 2024-09-21T15:34:53.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:34:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:34:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.294264665, 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:34:53.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.290288642999997, 2024-09-21T15:34:53.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:34:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.975 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.290262620999997, 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:34:53.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:34:53.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.290239067999998, 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.980 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.290204226, 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.984 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.988 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.286222703999996, 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.286115326999997, 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:34:53.996 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.286088757999998, 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:53.997 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:34:53.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:34:54.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:54.001 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.286061181999997, 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:54.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.005 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.286037671999999, 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.006 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:34:54.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282336045999998, 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.010 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:34:54.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:34:54.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:34:54.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282306292999998, 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.014 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:34:54.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.018 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282279224, 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.019 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.022 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282255888000002, 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.023 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.026 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282187284000003, 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:34:54.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:34:54.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.282158543999998, 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.031 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:34:54.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.278245560999999, 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.035 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:34:54.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:34:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:34:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 45.278190488999996, 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:34:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:34:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:34:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.043 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:54.043 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:53.970+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:54.044 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:54.052 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:54.050+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:54.153 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:54.150+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:54.170 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:54.170 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:54.170 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:54.170 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:54.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:54.254 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:54.250+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:54.303 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:54.303 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:54.303 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:54.303 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:54.303 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:54.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:54.355 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:54.354+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:54.439 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.659532503000001, 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:54.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.443 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.659154512000001, 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.444 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.658935257000003, 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.448 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:54.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:54.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.658538094000001, 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:54.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:54.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:54.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.658134306000001, 2024-09-21T15:34:54.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:54.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:54.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.461 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.657678543999999, 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:54.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:54.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.650451998000001, 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:54.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:54.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.650085709999999, 2024-09-21T15:34:54.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:54.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:54.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.474 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.649373867000001, 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:54.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.646113319999998, 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:54.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:54.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.482 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.645673101, 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:54.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.645361678, 2024-09-21T15:34:54.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.620434070000002, 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.496 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.619804414000001, 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:54.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:54.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.619399950999998, 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.505 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.618861557999999, 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.509 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.618334539999999, 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:54.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:54.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.617912814999997, 2024-09-21T15:34:54.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:54.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.518 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.617497413000002, 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.522 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 59.616946986000002, 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:54.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:54.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:54.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:54.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:54.527 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:54.528 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:54.523+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:54.529 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:54.630 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:54.627+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:54.663 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:54.663 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:54.663 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:54.663 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:54.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:54.730 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:54.727+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:54.796 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:54.796 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:54.796 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:54.796 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:54.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:54.831 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:54.827+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:54.906 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:34:54.911 INFO:teuthology.orchestra.run.smithi107.stderr:clear 3.10 pg_upmap mapping 2024-09-21T15:34:54.935 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:54.931+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187874 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391994453000001, 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089103661000000001, 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187874 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:54.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550069+0000", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550082+0000", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3054000000000001e-05 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550099+0000", 2024-09-21T15:34:54.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7122999999999999e-05 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550113+0000", 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3526999999999999e-05 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.638958+0000", 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088845460000000001 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639075+0000", 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117033 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639174+0000", 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8381000000000006e-05 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187875 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391939636, 2024-09-21T15:34:54.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089423839000000005, 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187875 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550124+0000", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550131+0000", 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7299999999999999e-06 2024-09-21T15:34:54.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550148+0000", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6909e-05 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550164+0000", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6010000000000001e-05 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639254+0000", 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089090091999999996 2024-09-21T15:34:54.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639387+0000", 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013300900000000001 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639549+0000", 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016201900000000001 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.951 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187876 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391888771, 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089634236000000006, 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187876 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550174+0000", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550181+0000", 2024-09-21T15:34:54.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1870000000000002e-06 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550197+0000", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6617e-05 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550213+0000", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5566e-05 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639619+0000", 2024-09-21T15:34:54.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089406157 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639711+0000", 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2303e-05 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639810+0000", 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8765999999999996e-05 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187877 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391839781, 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089883745000000001, 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187877 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550224+0000", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550230+0000", 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550247+0000", 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7445999999999999e-05 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550261+0000", 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.314e-05 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639864+0000", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089603631000000003 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639969+0000", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010485200000000001 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640109+0000", 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013950200000000001 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.959 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187878 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391791675, 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090091083000000002, 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187878 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550272+0000", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550278+0000", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550295+0000", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7037e-05 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550308+0000", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.306e-05 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640175+0000", 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089867531 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640269+0000", 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3774e-05 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640364+0000", 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4840999999999995e-05 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187879 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391745642, 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090330875000000005, 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187879 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550318+0000", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550324+0000", 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1929999999999998e-06 2024-09-21T15:34:54.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550341+0000", 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7180000000000002e-05 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550354+0000", 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3074000000000001e-05 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640420+0000", 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090065509000000002 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640517+0000", 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6803000000000001e-05 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640650+0000", 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013306900000000001 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187880 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391698069, 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090571654000000001, 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187880 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.968 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550366+0000", 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550371+0000", 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8939999999999998e-06 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550388+0000", 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6745999999999999e-05 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550402+0000", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.329e-05 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640716+0000", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090314426000000003 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640813+0000", 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7497000000000005e-05 2024-09-21T15:34:54.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640938+0000", 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124771 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187881 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391651982999999, 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090817446999999996, 2024-09-21T15:34:54.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187881 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550412+0000", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550418+0000", 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2929999999999997e-06 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550435+0000", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7073000000000001e-05 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550448+0000", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640992+0000", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090544168999999994 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641103+0000", 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110806 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641230+0000", 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012683200000000001 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187882 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391604518999999, 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090998446999999996, 2024-09-21T15:34:54.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187882 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550459+0000", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550465+0000", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8560000000000003e-06 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550482+0000", 2024-09-21T15:34:54.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6702999999999999e-05 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550494+0000", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2887e-05 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641273+0000", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090778677000000002 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641344+0000", 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1348999999999997e-05 2024-09-21T15:34:54.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641459+0000", 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114135 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187883 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391537014000001, 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091270417000000006, 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.980 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187883 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550526+0000", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550533+0000", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1069999999999999e-06 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550552+0000", 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8646000000000001e-05 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550568+0000", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5967000000000001e-05 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641512+0000", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090943916 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641657+0000", 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014500900000000001 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641798+0000", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014126500000000001 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187884 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.391484327000001, 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091471644000000005, 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.984 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187884 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550579+0000", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550585+0000", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2029999999999997e-06 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550603+0000", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7430000000000001e-05 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550616+0000", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3623e-05 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641866+0000", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091249446999999997 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641959+0000", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3412999999999995e-05 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642052+0000", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2731000000000004e-05 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187885 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.390920191999999, 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091213787000000005, 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187885 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550627+0000", 2024-09-21T15:34:54.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9994826 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551154+0000", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052656199999999995 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551178+0000", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4273e-05 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551195+0000", 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7499999999999998e-05 2024-09-21T15:34:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642107+0000", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090911480000000003 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642219+0000", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112209 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642358+0000", 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013906199999999999 2024-09-21T15:34:54.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187886 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.390853709, 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091413275000000002, 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187886 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.992 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551210+0000", 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551277+0000", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7926e-05 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551299+0000", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1759999999999998e-05 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551315+0000", 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5489e-05 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642432+0000", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091117087999999999 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642523+0000", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1644999999999994e-05 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642624+0000", 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100777 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187887 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.389163524000001, 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089972300000000005, 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187887 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551327+0000", 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9984255 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552910+0000", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015832330000000001 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552937+0000", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7183e-05 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552955+0000", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7629e-05 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:54.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642660+0000", 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089705650999999997 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642776+0000", 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116109 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642873+0000", 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7010999999999997e-05 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:54.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187888 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 593920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.389093651, 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090280418000000001, 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187888 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552970+0000", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552977+0000", 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4460000000000004e-06 2024-09-21T15:34:55.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552995+0000", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8386999999999999e-05 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553009+0000", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3597e-05 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642950+0000", 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089941260999999995 2024-09-21T15:34:55.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643087+0000", 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013637499999999999 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643251+0000", 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016482199999999999 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187889 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 618496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.003 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.389044290999999, 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090483776000000002, 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187889 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553019+0000", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553026+0000", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1229999999999997e-06 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553072+0000", 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6588999999999997e-05 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553088+0000", 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.558e-05 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643322+0000", 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090234272000000004 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643411+0000", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9073999999999995e-05 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643504+0000", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3084999999999994e-05 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187890 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 643072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:55.007 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.388959216, 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090643425, 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187890 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553104+0000", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553111+0000", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553130+0000", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.836e-05 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553147+0000", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7750000000000001e-05 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643540+0000", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090392792999999999 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643655+0000", 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011505700000000001 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643749+0000", 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3638e-05 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187900 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 888832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.38457947, 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089230480000000001, 2024-09-21T15:34:55.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187900 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:55.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557484+0000", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557490+0000", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557508+0000", 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7363999999999998e-05 2024-09-21T15:34:55.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557521+0000", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.362e-05 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646492+0000", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088970448999999993 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646603+0000", 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011123199999999999 2024-09-21T15:34:55.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646716+0000", 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112759 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187901 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.384532007000001, 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089469512000000001, 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.015 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187901 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557532+0000", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557537+0000", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7230000000000001e-06 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557555+0000", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7547e-05 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557568+0000", 2024-09-21T15:34:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2945999999999999e-05 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646770+0000", 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089201761000000004 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646869+0000", 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9148999999999995e-05 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647002+0000", 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013329600000000001 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187902 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 10.384484687, 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089715839000000006, 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187902 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557579+0000", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557585+0000", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.913e-06 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557602+0000", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7603000000000001e-05 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557616+0000", 2024-09-21T15:34:55.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3713000000000001e-05 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647074+0000", 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089457866999999996 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647182+0000", 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107559 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647296+0000", 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011416099999999999 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.023 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:55.024 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:55.035 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:55.027+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:55.133 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:55.131+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:55.189 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:55.189 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:55.189 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:55.189 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:55.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:55.252 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:55.250+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:55.324 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:55.324 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:55.334 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:55.330+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:55.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:55.435 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:55.431+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:55.500 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 100.376447164, 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:55.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:55.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.934290950999994, 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:55.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:55.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:55.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.934266205, 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:55.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:55.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.516 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.934221254999997, 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:55.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.520 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930364596999993, 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:55.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:55.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930320237000004, 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:55.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930266888000006, 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:55.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930235495000005, 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:55.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930189768999995, 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:55.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:55.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930160396000005, 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:55.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930135836000005, 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.930109199, 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:55.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:55.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:55.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.555 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926270631999998, 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:55.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926240812000003, 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.564 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926215282000001, 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:55.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.568 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926189575999999, 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:55.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926139535999994, 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:55.577 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926104735999999, 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:55.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:55.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.581 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.926080087000003, 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:55.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 75.922029679999994, 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:55.586 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:55.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:55.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:55.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:55.590 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:55.591 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:34:55.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:55.593 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:55.590+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:55.694 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:55.691+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:55.767 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:55.767 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:55.767 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:55.768 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:55.778 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:55.778 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:55.781 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:34:55.794 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:55.795 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:55.790+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:55.810 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:55.810 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:55.826 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:55.826 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:55.846 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:55.846 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:55.865 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:55.885 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:55.892 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:55.893 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:55.895 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:55.890+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:55.905 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:55.906 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:55.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:34:56.012 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:56.010+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:56.090 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.091 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.096 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:56.094+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:56.106 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:34:56.197 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:56.195+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:56.270 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.270 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.284 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:34:56.328 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:56.326+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:56.399 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:56.395+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.399 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.414 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:34:56.499 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:56.495+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:56.579 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.579 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.593 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:34:56.599 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:56.598+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:56.701 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:56.695+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:56.724 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.724 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.738 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:34:56.825 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:56.823+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:56.902 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:56.898+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:56.906 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:56.906 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:56.920 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:34:56.941 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:34:57.009 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:57.007+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:57.085 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.085 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:34:57.141 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:57.139+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:57.205 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:57.202+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:57.283 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.284 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.301 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:34:57.306 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:57.302+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:57.406 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:57.403+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:57.443 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.443 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:34:57.545 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:57.543+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:57.607 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:57.603+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:57.624 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.625 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.639 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:34:57.708 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:57.706+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:57.805 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.805 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:34:57.852 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:57.803+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:57.909 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:57.907+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:57.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:34:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:34:57.983 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:57.983 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:57.998 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:34:58.024 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:58.022+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:58.087 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:34:58.087 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:34:58.104 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:34:58.104 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:34:58.110 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:58.106+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:58.121 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:34:58.121 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:34:58.136 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:34:58.136 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:34:58.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:34:58.156 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:34:58.170 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:34:58.170 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:34:58.175 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:34:58.175 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:34:58.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:34:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:34:58.194 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:34:58.211 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:58.210+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:58.214 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:34:58.214 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:34:58.304 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:58.304 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:34:58.304 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:58.312 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:58.307+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:58.318 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:34:58.413 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:58.410+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:58.484 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:58.484 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:34:58.484 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:34:58.484 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:34:58.484 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:58.498 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:34:58.545 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:58.543+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:58.614 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:58.610+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:58.624 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:34:58.624 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:34:58.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:34:58.624 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.844768004000002, 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.625 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:34:58.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:34:58.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.844390013000002, 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:34:58.629 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:34:58.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.633 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.844170757999997, 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.635 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:34:58.637 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.843773595000002, 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.638 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:58.642 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.843369807000002, 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:58.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:34:58.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.646 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.842914045000001, 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.835687499000002, 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.651 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:34:58.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:34:58.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:34:58.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.835321211, 2024-09-21T15:34:58.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:34:58.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:34:58.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.662 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.834609368000002, 2024-09-21T15:34:58.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:34:58.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:34:58.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:34:58.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:34:58.671 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.672 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.831348820999999, 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.673 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:58.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:34:58.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:34:58.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:34:58.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.681 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.830908602000001, 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.683 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.684 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.689 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.830597179000002, 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.690 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:58.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:34:58.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:34:58.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:34:58.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:34:58.695 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.805669571000003, 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:34:58.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:34:58.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.805039915000002, 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.701 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.804635451999999, 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.705 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.708 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:58.709 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.804097059, 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:58.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.712 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.803570041, 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:34:58.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:34:58.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:34:58.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.803148315999998, 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:34:58.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:34:58.725 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.726 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.802732914000003, 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.727 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:34:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 63.802182487000003, 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:34:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:34:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:34:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:34:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:34:58.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:34:58.745 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:34:58.748 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:58.743+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:58.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:34:58.848 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:34:58.843+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:34:58.881 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:58.881 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:58.881 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:58.881 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:58.895 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:34:58.948 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:58.946+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:59.022 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:59.022 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:59.036 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:34:59.049 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:34:59.047+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:34:59.149 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:34:59.146+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:34:59.205 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:59.205 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:59.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187874 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654671693999999, 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089103661000000001, 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.206 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187874 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550069+0000", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550082+0000", 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3054000000000001e-05 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550099+0000", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7122999999999999e-05 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550113+0000", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3526999999999999e-05 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.638958+0000", 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088845460000000001 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639075+0000", 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117033 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639174+0000", 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8381000000000006e-05 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187875 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654616877, 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089423839000000005, 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187875 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550124+0000", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550131+0000", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7299999999999999e-06 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550148+0000", 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6909e-05 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550164+0000", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6010000000000001e-05 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639254+0000", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089090091999999996 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639387+0000", 2024-09-21T15:34:59.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013300900000000001 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639549+0000", 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016201900000000001 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187876 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654566012, 2024-09-21T15:34:59.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089634236000000006, 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187876 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550174+0000", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550181+0000", 2024-09-21T15:34:59.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1870000000000002e-06 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550197+0000", 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6617e-05 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550213+0000", 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5566e-05 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639619+0000", 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089406157 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639711+0000", 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2303e-05 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639810+0000", 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8765999999999996e-05 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187877 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654517022, 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089883745000000001, 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.220 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187877 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:34:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550224+0000", 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550230+0000", 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550247+0000", 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7445999999999999e-05 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550261+0000", 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.314e-05 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639864+0000", 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089603631000000003 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639969+0000", 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010485200000000001 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640109+0000", 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013950200000000001 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187878 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654468916000001, 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090091083000000002, 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187878 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.225 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550272+0000", 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550278+0000", 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550295+0000", 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7037e-05 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550308+0000", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.306e-05 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640175+0000", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089867531 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640269+0000", 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3774e-05 2024-09-21T15:34:59.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640364+0000", 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4840999999999995e-05 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187879 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654422883000001, 2024-09-21T15:34:59.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090330875000000005, 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187879 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550318+0000", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550324+0000", 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1929999999999998e-06 2024-09-21T15:34:59.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550341+0000", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7180000000000002e-05 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550354+0000", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3074000000000001e-05 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640420+0000", 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090065509000000002 2024-09-21T15:34:59.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640517+0000", 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6803000000000001e-05 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640650+0000", 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013306900000000001 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187880 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654375310000001, 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090571654000000001, 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187880 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550366+0000", 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550371+0000", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8939999999999998e-06 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550388+0000", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6745999999999999e-05 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550402+0000", 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.329e-05 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640716+0000", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090314426000000003 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640813+0000", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7497000000000005e-05 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640938+0000", 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124771 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.237 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187881 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654329224, 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090817446999999996, 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187881 2024-09-21T15:34:59.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550412+0000", 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550418+0000", 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2929999999999997e-06 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550435+0000", 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7073000000000001e-05 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550448+0000", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640992+0000", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090544168999999994 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641103+0000", 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110806 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641230+0000", 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012683200000000001 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187882 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.65428176, 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090998446999999996, 2024-09-21T15:34:59.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187882 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550459+0000", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550465+0000", 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8560000000000003e-06 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550482+0000", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6702999999999999e-05 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550494+0000", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2887e-05 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641273+0000", 2024-09-21T15:34:59.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090778677000000002 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641344+0000", 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1348999999999997e-05 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641459+0000", 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114135 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187883 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654214254999999, 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091270417000000006, 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187883 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550526+0000", 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550533+0000", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1069999999999999e-06 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550552+0000", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8646000000000001e-05 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550568+0000", 2024-09-21T15:34:59.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5967000000000001e-05 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641512+0000", 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090943916 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641657+0000", 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014500900000000001 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641798+0000", 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014126500000000001 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187884 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.654161567999999, 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091471644000000005, 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187884 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550579+0000", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550585+0000", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2029999999999997e-06 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550603+0000", 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7430000000000001e-05 2024-09-21T15:34:59.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550616+0000", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3623e-05 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641866+0000", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091249446999999997 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641959+0000", 2024-09-21T15:34:59.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3412999999999995e-05 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642052+0000", 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2731000000000004e-05 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187885 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.653597433, 2024-09-21T15:34:59.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091213787000000005, 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187885 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550627+0000", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9994826 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551154+0000", 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052656199999999995 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551178+0000", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4273e-05 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551195+0000", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7499999999999998e-05 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642107+0000", 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090911480000000003 2024-09-21T15:34:59.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642219+0000", 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112209 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642358+0000", 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013906199999999999 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187886 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.65353095, 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091413275000000002, 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187886 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551210+0000", 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551277+0000", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7926e-05 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551299+0000", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1759999999999998e-05 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551315+0000", 2024-09-21T15:34:59.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5489e-05 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642432+0000", 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091117087999999999 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642523+0000", 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1644999999999994e-05 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642624+0000", 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100777 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187887 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.651840764999999, 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089972300000000005, 2024-09-21T15:34:59.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187887 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551327+0000", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9984255 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552910+0000", 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015832330000000001 2024-09-21T15:34:59.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552937+0000", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7183e-05 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552955+0000", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7629e-05 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642660+0000", 2024-09-21T15:34:59.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089705650999999997 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642776+0000", 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116109 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642873+0000", 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7010999999999997e-05 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187888 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 593920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.651770892, 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090280418000000001, 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187888 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552970+0000", 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552977+0000", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4460000000000004e-06 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552995+0000", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8386999999999999e-05 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553009+0000", 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3597e-05 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642950+0000", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089941260999999995 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643087+0000", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013637499999999999 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643251+0000", 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016482199999999999 2024-09-21T15:34:59.272 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187889 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 618496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.651721532, 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090483776000000002, 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187889 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553019+0000", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553026+0000", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1229999999999997e-06 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553072+0000", 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6588999999999997e-05 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553088+0000", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.558e-05 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643322+0000", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090234272000000004 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643411+0000", 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9073999999999995e-05 2024-09-21T15:34:59.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643504+0000", 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3084999999999994e-05 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187890 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 643072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.651636457, 2024-09-21T15:34:59.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090643425, 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187890 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553104+0000", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553111+0000", 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:34:59.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553130+0000", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.836e-05 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553147+0000", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7750000000000001e-05 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643540+0000", 2024-09-21T15:34:59.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090392792999999999 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643655+0000", 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011505700000000001 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643749+0000", 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3638e-05 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187900 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 888832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.647256711000001, 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089230480000000001, 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187900 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557484+0000", 2024-09-21T15:34:59.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557490+0000", 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557508+0000", 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7363999999999998e-05 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557521+0000", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.362e-05 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646492+0000", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088970448999999993 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646603+0000", 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011123199999999999 2024-09-21T15:34:59.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646716+0000", 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112759 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187901 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.647209247999999, 2024-09-21T15:34:59.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089469512000000001, 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187901 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557532+0000", 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557537+0000", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7230000000000001e-06 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557555+0000", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7547e-05 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557568+0000", 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2945999999999999e-05 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646770+0000", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089201761000000004 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646869+0000", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9148999999999995e-05 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647002+0000", 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013329600000000001 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187902 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 14.647161927999999, 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089715839000000006, 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187902 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557579+0000", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557585+0000", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.913e-06 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557602+0000", 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7603000000000001e-05 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557616+0000", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3713000000000001e-05 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647074+0000", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089457866999999996 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647182+0000", 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107559 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647296+0000", 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011416099999999999 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.295 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:59.296 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:34:59.297 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:59.294+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:59.398 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:34:59.394+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:34:59.466 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:59.466 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:59.466 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:34:59.466 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:59.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:34:59.498 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:34:59.495+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:34:59.599 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:59.595+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:59.649 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:59.649 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:34:59.649 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:34:59.649 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:34:59.649 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:59.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:34:59.700 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:34:59.695+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:59.787 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 104.662039061, 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.789 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:34:59.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:34:59.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:34:59.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.219882847999997, 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:34:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.798 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.219858102000003, 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:59.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:34:59.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.219813152, 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.803 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.805 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:34:59.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215956493999997, 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:34:59.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:34:59.811 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215912134000007, 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:34:59.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.816 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215858784999995, 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:59.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:34:59.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:34:59.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215827391999994, 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:34:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215781665999998, 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:34:59.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:34:59.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215752292999994, 2024-09-21T15:34:59.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:34:59.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:34:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215727732999994, 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:34:59.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:34:59.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.215701096000004, 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:34:59.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:34:59.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211862529000001, 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.844 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:34:59.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211832709000007, 2024-09-21T15:34:59.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:34:59.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211807179000004, 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:34:59.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:34:59.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211781473000002, 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:34:59.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211731432999997, 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.861 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:34:59.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:34:59.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211696633000003, 2024-09-21T15:34:59.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:34:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.211671984000006, 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.870 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:34:59.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:34:59.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.873 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 80.207621576999998, 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.874 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:34:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:34:59.877 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:34:59.878 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:34:59.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:34:59.879 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:34:59.874+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:34:59.979 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:34:59.974+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:34:59.981 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:35:00.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:00.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:00.025 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T15:35:00.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:35:00.060 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:00.060 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:00.060 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:00.079 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:00.078+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:00.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:00.180 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:00.178+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:00.233 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:00.244 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:00.244 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:00.244 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:00.245 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:00.245 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:00.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:00.281 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:00.279+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 118, 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:34:59.958805+0000", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:33:20.030364+0000", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:35:00.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5801984, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "noscrub", 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:35:00.343 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "luminous", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:00.344 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:00.345 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:00.346 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:00.347 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:00.348 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:00.349 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:00.350 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:00.351 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:00.352 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:00.353 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:00.354 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:35:00.355 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:00.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:00.357 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "118", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:00.358 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 49, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 118, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:00.359 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:35:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 101, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.363 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.364 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.368 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 97, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.371 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.373 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 89, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:35:00.375 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.377 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:35:00.378 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.379 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 100, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 88, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.385 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.386 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.387 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:35:00.388 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:35:00.389 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 115, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.391 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.392 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:35:00.395 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 116, 2024-09-21T15:35:00.396 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.398 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:35:00.399 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.400 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 101, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:00.402 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.404 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.405 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.406 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.408 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:35:00.409 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.410 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [ 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.14", 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: 2, 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: 6 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.414 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:35:00.415 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [ 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 40, 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 5 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.416 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [ 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 40, 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 1 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.417 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:00.418 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:00.420 INFO:tasks.thrashosds.thrasher:j is {'epoch': 118, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:34:59.958805+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:33:20.030364+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 49, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 101, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.14', 'osds': [2, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 40, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 40, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:35:00.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:35:00.421 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:00.421 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:00.421 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:00.422 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:00.422 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:00.422 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:00.423 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:00.423 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:00.423 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:00.415+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:00.424 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:00.425 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:00.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:00.444 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:00.445 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:00.464 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:00.483 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:00.483 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:00.502 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:00.503 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:00.520 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:00.515+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:00.556 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:00.556 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:00.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:00.621 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:00.619+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:00.705 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:35:00.706 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:35:00.721 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:00.721 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:00.722 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-21T15:34:59.967225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":874428192,"num_objects":1375,"num_object_clones":305,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":476,"num_read":235818,"num_read_kb":496139,"num_write":16016,"num_write_kb":383697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":23936968,"num_keys_recovered":139,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12821,"ondisk_log_size":12821,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515756,"kb_used_data":196268,"kb_used_omap":12,"kb_used_meta":319347,"kb_avail":754458964,"statfs":{"total":773094113280,"available":772565979136,"internally_reserved":0,"allocated":200978432,"data_stored":231686477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":327011897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-135816914,"num_objects":-111,"num_object_clones":-43,"num_object_copies":-222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-111,"num_whiteouts":-1,"num_read":774,"num_read_kb":662,"num_write":170,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":3569869,"num_keys_recovered":12,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.042420"},"pg_stats":[{"pgid":"3.1f","version":"116'628","reported_seq":5159,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.070648+0000","last_change":"2024-09-21T15:34:57.956230+0000","last_active":"2024-09-21T15:34:58.070648+0000","last_peered":"2024-09-21T15:34:58.070648+0000","last_clean":"2024-09-21T15:34:58.070648+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:34:58.070648+0000","last_undegraded":"2024-09-21T15:34:58.070648+0000","last_fullsized":"2024-09-21T15:34:58.070648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:41:51.716535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040393039999999996,"stat_sum":{"num_bytes":22084284,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":6951,"num_read_kb":8278,"num_write":961,"num_write_kb":3270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1e","version":"116'431","reported_seq":7024,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.074967+0000","last_change":"2024-09-21T15:34:57.959650+0000","last_active":"2024-09-21T15:34:58.074967+0000","last_peered":"2024-09-21T15:34:58.074967+0000","last_clean":"2024-09-21T15:34:58.074967+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:34:58.074967+0000","last_undegraded":"2024-09-21T15:34:58.074967+0000","last_fullsized":"2024-09-21T15:34:58.074967+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003656397,"stat_sum":{"num_bytes":31764946,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":8522,"num_read_kb":21647,"num_write":552,"num_write_kb":26505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1d","version":"113'147","reported_seq":484,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.960766+0000","last_change":"2024-09-21T15:34:57.961883+0000","last_active":"2024-09-21T15:34:57.960766+0000","last_peered":"2024-09-21T15:34:57.960766+0000","last_clean":"2024-09-21T15:34:57.960766+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:34:57.960766+0000","last_undegraded":"2024-09-21T15:34:57.960766+0000","last_fullsized":"2024-09-21T15:34:57.960766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002329888,"stat_sum":{"num_bytes":33678683,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":283,"num_read_kb":6246,"num_write":199,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1c","version":"116'264","reported_seq":5638,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.063677+0000","last_change":"2024-09-21T15:34:57.959936+0000","last_active":"2024-09-21T15:34:58.063677+0000","last_peered":"2024-09-21T15:34:58.063677+0000","last_clean":"2024-09-21T15:34:58.063677+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:34:58.063677+0000","last_undegraded":"2024-09-21T15:34:58.063677+0000","last_fullsized":"2024-09-21T15:34:58.063677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014566830000000001,"stat_sum":{"num_bytes":47697616,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":5481,"num_read_kb":22909,"num_write":271,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1b","version":"115'358","reported_seq":9472,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.955213+0000","last_change":"2024-09-21T15:34:57.955717+0000","last_active":"2024-09-21T15:34:57.955213+0000","last_peered":"2024-09-21T15:34:57.955213+0000","last_clean":"2024-09-21T15:34:57.955213+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:34:57.955213+0000","last_undegraded":"2024-09-21T15:34:57.955213+0000","last_fullsized":"2024-09-21T15:34:57.955213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:54.706148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002825648,"stat_sum":{"num_bytes":17839064,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":10153,"num_read_kb":17274,"num_write":344,"num_write_kb":7281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1a","version":"116'217","reported_seq":6828,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.096406+0000","last_change":"2024-09-21T15:34:57.980688+0000","last_active":"2024-09-21T15:34:58.096406+0000","last_peered":"2024-09-21T15:34:58.096406+0000","last_clean":"2024-09-21T15:34:58.096406+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:34:58.096406+0000","last_undegraded":"2024-09-21T15:34:58.096406+0000","last_fullsized":"2024-09-21T15:34:58.096406+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067946900000000004,"stat_sum":{"num_bytes":18286207,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":6609,"num_read_kb":35814,"num_write":216,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.19","version":"116'578","reported_seq":33313,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.033723+0000","last_change":"2024-09-21T15:34:58.033723+0000","last_active":"2024-09-21T15:34:58.033723+0000","last_peered":"2024-09-21T15:34:58.033723+0000","last_clean":"2024-09-21T15:34:58.033723+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:34:58.033723+0000","last_undegraded":"2024-09-21T15:34:58.033723+0000","last_fullsized":"2024-09-21T15:34:58.033723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001338655,"stat_sum":{"num_bytes":64439773,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":34622,"num_read_kb":22737,"num_write":809,"num_write_kb":6873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.18","version":"111'406","reported_seq":961,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.954536+0000","last_change":"2024-09-21T15:34:57.955039+0000","last_active":"2024-09-21T15:34:57.954536+0000","last_peered":"2024-09-21T15:34:57.954536+0000","last_clean":"2024-09-21T15:34:57.954536+0000","last_became_active":"2024-09-21T15:33:03.921578+0000","last_became_peered":"2024-09-21T15:33:03.921578+0000","last_unstale":"2024-09-21T15:34:57.954536+0000","last_undegraded":"2024-09-21T15:34:57.954536+0000","last_fullsized":"2024-09-21T15:34:57.954536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:46.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029490620000000001,"stat_sum":{"num_bytes":17567475,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":2055,"num_read_kb":4687,"num_write":567,"num_write_kb":13264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.17","version":"110'597","reported_seq":1839,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.955782+0000","last_change":"2024-09-21T15:34:57.956239+0000","last_active":"2024-09-21T15:34:57.955782+0000","last_peered":"2024-09-21T15:34:57.955782+0000","last_clean":"2024-09-21T15:34:57.955782+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:34:57.955782+0000","last_undegraded":"2024-09-21T15:34:57.955782+0000","last_fullsized":"2024-09-21T15:34:57.955782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:09.288430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028545250000000001,"stat_sum":{"num_bytes":29991782,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3863,"num_read_kb":11473,"num_write":905,"num_write_kb":7862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.16","version":"113'312","reported_seq":5808,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.958618+0000","last_change":"2024-09-21T15:34:57.959738+0000","last_active":"2024-09-21T15:34:57.958618+0000","last_peered":"2024-09-21T15:34:57.958618+0000","last_clean":"2024-09-21T15:34:57.958618+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:34:57.958618+0000","last_undegraded":"2024-09-21T15:34:57.958618+0000","last_fullsized":"2024-09-21T15:34:57.958618+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001273592,"stat_sum":{"num_bytes":27778453,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":6963,"num_read_kb":11462,"num_write":313,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.15","version":"113'272","reported_seq":5051,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.951399+0000","last_change":"2024-09-21T15:34:57.952231+0000","last_active":"2024-09-21T15:34:57.951399+0000","last_peered":"2024-09-21T15:34:57.951399+0000","last_clean":"2024-09-21T15:34:57.951399+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:34:57.951399+0000","last_undegraded":"2024-09-21T15:34:57.951399+0000","last_fullsized":"2024-09-21T15:34:57.951399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014877829999999999,"stat_sum":{"num_bytes":14420493,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":4842,"num_read_kb":14908,"num_write":308,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.14","version":"116'580","reported_seq":4099,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.063751+0000","last_change":"2024-09-21T15:34:58.063751+0000","last_active":"2024-09-21T15:34:58.063751+0000","last_peered":"2024-09-21T15:34:58.063751+0000","last_clean":"2024-09-21T15:34:58.063751+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:34:58.063751+0000","last_undegraded":"2024-09-21T15:34:58.063751+0000","last_fullsized":"2024-09-21T15:34:58.063751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010708460000000001,"stat_sum":{"num_bytes":28098524,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":5922,"num_read_kb":15458,"num_write":725,"num_write_kb":21346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.186089+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:34:57.186089+0000","last_peered":"2024-09-21T15:34:57.186089+0000","last_clean":"2024-09-21T15:34:57.186089+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:34:57.186089+0000","last_undegraded":"2024-09-21T15:34:57.186089+0000","last_fullsized":"2024-09-21T15:34:57.186089+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"116'438","reported_seq":4330,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.079375+0000","last_change":"2024-09-21T15:34:57.955104+0000","last_active":"2024-09-21T15:34:58.079375+0000","last_peered":"2024-09-21T15:34:58.079375+0000","last_clean":"2024-09-21T15:34:58.079375+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:34:58.079375+0000","last_undegraded":"2024-09-21T15:34:58.079375+0000","last_fullsized":"2024-09-21T15:34:58.079375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:04.212940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002894457,"stat_sum":{"num_bytes":36518282,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":7068,"num_read_kb":13417,"num_write":403,"num_write_kb":4706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":113,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.225870+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:34:57.225870+0000","last_peered":"2024-09-21T15:34:57.225870+0000","last_clean":"2024-09-21T15:34:57.225870+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:34:57.225870+0000","last_undegraded":"2024-09-21T15:34:57.225870+0000","last_fullsized":"2024-09-21T15:34:57.225870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"115'260","reported_seq":1329,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.952621+0000","last_change":"2024-09-21T15:34:57.953522+0000","last_active":"2024-09-21T15:34:57.952621+0000","last_peered":"2024-09-21T15:34:57.952621+0000","last_clean":"2024-09-21T15:34:57.952621+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:34:57.952621+0000","last_undegraded":"2024-09-21T15:34:57.952621+0000","last_fullsized":"2024-09-21T15:34:57.952621+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021916269999999998,"stat_sum":{"num_bytes":37229564,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":1726,"num_read_kb":23153,"num_write":329,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":113,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.933649+0000","last_change":"2024-09-21T15:32:39.777206+0000","last_active":"2024-09-21T15:34:56.933649+0000","last_peered":"2024-09-21T15:34:56.933649+0000","last_clean":"2024-09-21T15:34:56.933649+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:34:56.933649+0000","last_undegraded":"2024-09-21T15:34:56.933649+0000","last_fullsized":"2024-09-21T15:34:56.933649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:48.253617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"113'180","reported_seq":779,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.033783+0000","last_change":"2024-09-21T15:34:58.033959+0000","last_active":"2024-09-21T15:34:58.033783+0000","last_peered":"2024-09-21T15:34:58.033783+0000","last_clean":"2024-09-21T15:34:58.033783+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:34:58.033783+0000","last_undegraded":"2024-09-21T15:34:58.033783+0000","last_fullsized":"2024-09-21T15:34:58.033783+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372493,"stat_sum":{"num_bytes":18384196,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":836,"num_read_kb":4075,"num_write":184,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":113,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.225913+0000","last_change":"2024-09-21T15:32:38.790989+0000","last_active":"2024-09-21T15:34:57.225913+0000","last_peered":"2024-09-21T15:34:57.225913+0000","last_clean":"2024-09-21T15:34:57.225913+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:34:57.225913+0000","last_undegraded":"2024-09-21T15:34:57.225913+0000","last_fullsized":"2024-09-21T15:34:57.225913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.784298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"116'700","reported_seq":1151,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.113826+0000","last_change":"2024-09-21T15:34:57.962851+0000","last_active":"2024-09-21T15:34:58.113826+0000","last_peered":"2024-09-21T15:34:58.113826+0000","last_clean":"2024-09-21T15:34:58.113826+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:34:58.113826+0000","last_undegraded":"2024-09-21T15:34:58.113826+0000","last_fullsized":"2024-09-21T15:34:58.113826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031306839999999999,"stat_sum":{"num_bytes":13447877,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":2979,"num_read_kb":10410,"num_write":1001,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.3","version":"116'257","reported_seq":6126,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.083806+0000","last_change":"2024-09-21T15:34:57.980911+0000","last_active":"2024-09-21T15:34:58.083806+0000","last_peered":"2024-09-21T15:34:58.083806+0000","last_clean":"2024-09-21T15:34:58.083806+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:34:58.083806+0000","last_undegraded":"2024-09-21T15:34:58.083806+0000","last_fullsized":"2024-09-21T15:34:58.083806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:04.220813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067824400000000005,"stat_sum":{"num_bytes":24593565,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":5855,"num_read_kb":16396,"num_write":273,"num_write_kb":6737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.2","version":"20'2","reported_seq":114,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.929715+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:34:56.929715+0000","last_peered":"2024-09-21T15:34:56.929715+0000","last_clean":"2024-09-21T15:34:56.929715+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:34:56.929715+0000","last_undegraded":"2024-09-21T15:34:56.929715+0000","last_fullsized":"2024-09-21T15:34:56.929715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"113'756","reported_seq":2266,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.032538+0000","last_change":"2024-09-21T15:34:58.032705+0000","last_active":"2024-09-21T15:34:58.032538+0000","last_peered":"2024-09-21T15:34:58.032538+0000","last_clean":"2024-09-21T15:34:58.032538+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:34:58.032538+0000","last_undegraded":"2024-09-21T15:34:58.032538+0000","last_fullsized":"2024-09-21T15:34:58.032538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034275700000000001,"stat_sum":{"num_bytes":26067297,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":5366,"num_read_kb":8681,"num_write":1036,"num_write_kb":11831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.931737+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:34:56.931737+0000","last_peered":"2024-09-21T15:34:56.931737+0000","last_clean":"2024-09-21T15:34:56.931737+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:34:56.931737+0000","last_undegraded":"2024-09-21T15:34:56.931737+0000","last_fullsized":"2024-09-21T15:34:56.931737+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"113'457","reported_seq":2681,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.096822+0000","last_change":"2024-09-21T15:34:57.954334+0000","last_active":"2024-09-21T15:34:58.096822+0000","last_peered":"2024-09-21T15:34:58.096822+0000","last_clean":"2024-09-21T15:34:58.096822+0000","last_became_active":"2024-09-21T15:34:29.762828+0000","last_became_peered":"2024-09-21T15:34:29.762828+0000","last_unstale":"2024-09-21T15:34:58.096822+0000","last_undegraded":"2024-09-21T15:34:58.096822+0000","last_fullsized":"2024-09-21T15:34:58.096822+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:17.503095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00258735,"stat_sum":{"num_bytes":25830907,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":11,"num_read":3862,"num_read_kb":12323,"num_write":650,"num_write_kb":13927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":113,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.934351+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:34:56.934351+0000","last_peered":"2024-09-21T15:34:56.934351+0000","last_clean":"2024-09-21T15:34:56.934351+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:34:56.934351+0000","last_undegraded":"2024-09-21T15:34:56.934351+0000","last_fullsized":"2024-09-21T15:34:56.934351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"113'335","reported_seq":875,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.960071+0000","last_change":"2024-09-21T15:34:57.960606+0000","last_active":"2024-09-21T15:34:57.960071+0000","last_peered":"2024-09-21T15:34:57.960071+0000","last_clean":"2024-09-21T15:34:57.960071+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:34:57.960071+0000","last_undegraded":"2024-09-21T15:34:57.960071+0000","last_fullsized":"2024-09-21T15:34:57.960071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014116230000000001,"stat_sum":{"num_bytes":24646899,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":25,"num_read":1471,"num_read_kb":4695,"num_write":374,"num_write_kb":6517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.3","version":"18'1","reported_seq":137,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.929986+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:34:56.929986+0000","last_peered":"2024-09-21T15:34:56.929986+0000","last_clean":"2024-09-21T15:34:56.929986+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:34:56.929986+0000","last_undegraded":"2024-09-21T15:34:56.929986+0000","last_fullsized":"2024-09-21T15:34:56.929986+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T15:34:56.933409+0000","last_change":"2024-09-21T15:34:33.800946+0000","last_active":"2024-09-21T15:34:56.933409+0000","last_peered":"2024-09-21T15:34:56.933409+0000","last_clean":"2024-09-21T15:34:56.933409+0000","last_became_active":"2024-09-21T15:34:33.799956+0000","last_became_peered":"2024-09-21T15:34:33.799956+0000","last_unstale":"2024-09-21T15:34:56.933409+0000","last_undegraded":"2024-09-21T15:34:56.933409+0000","last_fullsized":"2024-09-21T15:34:56.933409+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:57.645467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"116'539","reported_seq":1488,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.110659+0000","last_change":"2024-09-21T15:34:58.033874+0000","last_active":"2024-09-21T15:34:58.110659+0000","last_peered":"2024-09-21T15:34:58.110659+0000","last_clean":"2024-09-21T15:34:58.110659+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:34:58.110659+0000","last_undegraded":"2024-09-21T15:34:58.110659+0000","last_fullsized":"2024-09-21T15:34:58.110659+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014671129999999999,"stat_sum":{"num_bytes":12931104,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2904,"num_read_kb":7084,"num_write":768,"num_write_kb":5819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.9","version":"113'472","reported_seq":1555,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.980233+0000","last_change":"2024-09-21T15:34:57.980233+0000","last_active":"2024-09-21T15:34:57.980233+0000","last_peered":"2024-09-21T15:34:57.980233+0000","last_clean":"2024-09-21T15:34:57.980233+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:34:57.980233+0000","last_undegraded":"2024-09-21T15:34:57.980233+0000","last_fullsized":"2024-09-21T15:34:57.980233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046006999999999998,"stat_sum":{"num_bytes":24522187,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":3136,"num_read_kb":7866,"num_write":631,"num_write_kb":6149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.a","version":"116'288","reported_seq":4807,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.068594+0000","last_change":"2024-09-21T15:34:57.955848+0000","last_active":"2024-09-21T15:34:58.068594+0000","last_peered":"2024-09-21T15:34:58.068594+0000","last_clean":"2024-09-21T15:34:58.068594+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:34:58.068594+0000","last_undegraded":"2024-09-21T15:34:58.068594+0000","last_fullsized":"2024-09-21T15:34:58.068594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072100599999999995,"stat_sum":{"num_bytes":21750935,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4762,"num_read_kb":15787,"num_write":308,"num_write_kb":6694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.b","version":"116'341","reported_seq":8947,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.955600+0000","last_change":"2024-09-21T15:34:57.955600+0000","last_active":"2024-09-21T15:34:57.955600+0000","last_peered":"2024-09-21T15:34:57.955600+0000","last_clean":"2024-09-21T15:34:57.955600+0000","last_became_active":"2024-09-21T15:33:03.918908+0000","last_became_peered":"2024-09-21T15:33:03.918908+0000","last_unstale":"2024-09-21T15:34:57.955600+0000","last_undegraded":"2024-09-21T15:34:57.955600+0000","last_fullsized":"2024-09-21T15:34:57.955600+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:28.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032999309999999999,"stat_sum":{"num_bytes":33161860,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":10265,"num_read_kb":20513,"num_write":360,"num_write_kb":11076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.c","version":"113'254","reported_seq":2019,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.959908+0000","last_change":"2024-09-21T15:34:57.961010+0000","last_active":"2024-09-21T15:34:57.959908+0000","last_peered":"2024-09-21T15:34:57.959908+0000","last_clean":"2024-09-21T15:34:57.959908+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:34:57.959908+0000","last_undegraded":"2024-09-21T15:34:57.959908+0000","last_fullsized":"2024-09-21T15:34:57.959908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024605069999999998,"stat_sum":{"num_bytes":37448049,"num_objects":45,"num_object_clones":17,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":1834,"num_read_kb":12523,"num_write":251,"num_write_kb":23583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.d","version":"113'356","reported_seq":17350,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.953839+0000","last_change":"2024-09-21T15:34:57.955060+0000","last_active":"2024-09-21T15:34:57.953839+0000","last_peered":"2024-09-21T15:34:57.953839+0000","last_clean":"2024-09-21T15:34:57.953839+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:34:57.953839+0000","last_undegraded":"2024-09-21T15:34:57.953839+0000","last_fullsized":"2024-09-21T15:34:57.953839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036074050000000002,"stat_sum":{"num_bytes":14446653,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":17984,"num_read_kb":7702,"num_write":504,"num_write_kb":13560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.e","version":"116'318","reported_seq":37821,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.952191+0000","last_change":"2024-09-21T15:34:57.952191+0000","last_active":"2024-09-21T15:34:57.952191+0000","last_peered":"2024-09-21T15:34:57.952191+0000","last_clean":"2024-09-21T15:34:57.952191+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:34:57.952191+0000","last_undegraded":"2024-09-21T15:34:57.952191+0000","last_fullsized":"2024-09-21T15:34:57.952191+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013766130000000001,"stat_sum":{"num_bytes":42565788,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":37992,"num_read_kb":47779,"num_write":346,"num_write_kb":22499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.f","version":"113'367","reported_seq":1680,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.952474+0000","last_change":"2024-09-21T15:34:57.952821+0000","last_active":"2024-09-21T15:34:57.952474+0000","last_peered":"2024-09-21T15:34:57.952474+0000","last_clean":"2024-09-21T15:34:57.952474+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:34:57.952474+0000","last_undegraded":"2024-09-21T15:34:57.952474+0000","last_fullsized":"2024-09-21T15:34:57.952474+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015156460000000001,"stat_sum":{"num_bytes":30043907,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":2567,"num_read_kb":8764,"num_write":445,"num_write_kb":5117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.10","version":"115'238","reported_seq":773,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-21T15:34:57.427982+0000","last_change":"2024-09-21T15:34:58.688909+0000","last_active":"2024-09-21T15:34:58.688909+0000","last_peered":"2024-09-21T15:34:57.427982+0000","last_clean":"2024-09-21T15:34:52.448481+0000","last_became_active":"2024-09-21T15:34:57.196415+0000","last_became_peered":"2024-09-21T15:34:57.196415+0000","last_unstale":"2024-09-21T15:34:57.427982+0000","last_undegraded":"2024-09-21T15:34:57.427982+0000","last_fullsized":"2024-09-21T15:34:56.929683+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:08:25.389554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052840899999999998,"stat_sum":{"num_bytes":18622143,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":989,"num_read_kb":7724,"num_write":226,"num_write_kb":6139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":4368836,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2c","length":"1"}]},{"pgid":"3.11","version":"116'461","reported_seq":6107,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.113138+0000","last_change":"2024-09-21T15:34:57.951164+0000","last_active":"2024-09-21T15:34:58.113138+0000","last_peered":"2024-09-21T15:34:58.113138+0000","last_clean":"2024-09-21T15:34:58.113138+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:34:58.113138+0000","last_undegraded":"2024-09-21T15:34:58.113138+0000","last_fullsized":"2024-09-21T15:34:58.113138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011009450000000001,"stat_sum":{"num_bytes":27596605,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":8285,"num_read_kb":22799,"num_write":590,"num_write_kb":19165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.12","version":"113'525","reported_seq":7253,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:57.953805+0000","last_change":"2024-09-21T15:34:57.954369+0000","last_active":"2024-09-21T15:34:57.953805+0000","last_peered":"2024-09-21T15:34:57.953805+0000","last_clean":"2024-09-21T15:34:57.953805+0000","last_became_active":"2024-09-21T15:34:33.798598+0000","last_became_peered":"2024-09-21T15:34:33.798598+0000","last_unstale":"2024-09-21T15:34:57.953805+0000","last_undegraded":"2024-09-21T15:34:57.953805+0000","last_fullsized":"2024-09-21T15:34:57.953805+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:18.729091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025680719999999998,"stat_sum":{"num_bytes":21295696,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":7086,"num_read_kb":18505,"num_write":664,"num_write_kb":22829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2138377,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.13","version":"116'369","reported_seq":10262,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T15:34:58.063919+0000","last_change":"2024-09-21T15:34:57.951124+0000","last_active":"2024-09-21T15:34:58.063919+0000","last_peered":"2024-09-21T15:34:58.063919+0000","last_clean":"2024-09-21T15:34:58.063919+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:34:58.063919+0000","last_undegraded":"2024-09-21T15:34:58.063919+0000","last_fullsized":"2024-09-21T15:34:58.063919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070935599999999996,"stat_sum":{"num_bytes":29086991,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":12469,"num_read_kb":32950,"num_write":342,"num_write_kb":15121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":873837805,"num_objects":1371,"num_object_clones":305,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":476,"num_read":235702,"num_read_kb":496039,"num_write":15855,"num_write_kb":381279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":884,"num_bytes_recovered":22756232,"num_keys_recovered":139,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188440576,"data_stored":225069445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12701,"ondisk_log_size":12701,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65616,"kb_used_data":34496,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94306224,"statfs":{"total":96636764160,"available":96569573376,"internally_reserved":0,"allocated":35323904,"data_stored":40359128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.67500000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.622}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84399999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.879}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.98399999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":1.0309999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.054},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.075}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":15812,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":16191488,"data_stored":17779994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.84799999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.52800000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.627}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74204,"kb_used_data":25040,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94297636,"statfs":{"total":96636764160,"available":96560779264,"internally_reserved":0,"allocated":25640960,"data_stored":29970886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":3.1419999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":0.90400000000000003}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":3.1709999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.96699999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":3.1000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65120,"kb_used_data":33996,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94306720,"statfs":{"total":96636764160,"available":96570081280,"internally_reserved":0,"allocated":34811904,"data_stored":38097157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.88500000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.71099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.75600000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.79400000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.1559999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0249999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":1.0640000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83044,"kb_used_data":37968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94288796,"statfs":{"total":96636764160,"available":96551727104,"internally_reserved":0,"allocated":38879232,"data_stored":47714881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60099999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.69199999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.83999999999999997},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":1.171}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.995}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.026}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.147}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42336,"kb_used_data":7124,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329504,"statfs":{"total":96636764160,"available":96593412096,"internally_reserved":0,"allocated":7294976,"data_stored":7583376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.1140000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.72799999999999998,"15min":0.75800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.613,"15min":1.613},"last":1.157},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89700000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.159,"15min":1.159},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90100000000000002,"15min":0.91800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1919999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.81699999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.536,"15min":1.536},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1699999999999999,"5min":1.52,"15min":1.52},"last":0.85299999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.94399999999999995,"15min":0.96999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0860000000000001,"5min":1.617,"15min":1.617},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.95199999999999996,"15min":0.97799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.995,"5min":1.554,"15min":1.554},"last":0.92400000000000004}]},{"osd":7,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1200000000000001,"5min":1.669,"15min":1.669},"last":1.089},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2390000000000001,"5min":1.589,"15min":1.589},"last":1.0309999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53000,"kb_used_data":13692,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94318840,"statfs":{"total":96636764160,"available":96582492160,"internally_reserved":0,"allocated":14020608,"data_stored":16233731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.81999999999999995,"15min":0.79600000000000004},"min":{"1min":0.55000000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.752},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.55900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2390000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2729999999999999,"5min":1.514,"15min":1.514},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.67200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.891,"15min":1.891},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.94099999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3480000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.98899999999999999,"15min":1.004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3919999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.95699999999999996,"15min":0.93999999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.67800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77308,"kb_used_data":28140,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94294532,"statfs":{"total":96636764160,"available":96557600768,"internally_reserved":0,"allocated":28815360,"data_stored":33947324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.28599999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.0329999999999999,"5min":1.397,"15min":1.397},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64800000000000002,"15min":0.64000000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.4870000000000001,"5min":3.4870000000000001,"15min":3.4870000000000001},"last":0.66400000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.016,"5min":1.23,"15min":1.23},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.038,"5min":1.133,"15min":1.133},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.0229999999999997,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78300000000000003,"15min":0.79100000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3120000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.48599999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2669999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.91200000000000003,"15min":0.92600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2110000000000001,"5min":1.867,"15min":1.867},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.95499999999999996,"15min":0.93799999999999994},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.99399999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0049999999999999,"15min":1.0169999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.464,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.55500000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2649999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.95099999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97499999999999998,"15min":0.97899999999999998},"min":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.427,"5min":1.804,"15min":1.804},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0169999999999999,"15min":1.0009999999999999},"min":{"1min":0.58899999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":1.165}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26836992,"data_stored":32708834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12632064,"data_stored":15585590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5910528,"data_stored":6935254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37494784,"data_stored":47066759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33427456,"data_stored":37449035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24203264,"data_stored":29276553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14798848,"data_stored":17122367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33136640,"data_stored":38925053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:35:00.725 INFO:tasks.thrashosds.thrasher:Setting 3.1a to [7, 3, 6, 0] 2024-09-21T15:35:00.725 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1a', '7', '3', '6', '0'] 2024-09-21T15:35:00.725 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1a 7 3 6 0 2024-09-21T15:35:00.725 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:00.718+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:00.735 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:00.824 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:00.822+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:00.891 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:00.891 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:00.904 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:00.947 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:00.946+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:00.974 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:35:01.024 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:01.019+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:01.047 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.047 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:01.154 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:01.151+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:01.225 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:01.223+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:01.229 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.229 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.243 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:01.326 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:01.322+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:01.407 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.407 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:01.465 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:01.463+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:01.528 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:01.526+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:01.543 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.543 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.557 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:01.629 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:01.626+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:01.722 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.722 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.730 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:01.727+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:01.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:01.756 INFO:teuthology.orchestra.run.smithi107.stderr:set 3.1a pg_upmap_items mapping to [7->3,6->0] 2024-09-21T15:35:01.830 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:01.827+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:01.905 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:01.936 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:01.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:01.938 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:01.934+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:02.039 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:02.034+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:02.104 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:02.118 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:02.139 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:02.138+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:02.240 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:02.240 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:02.240 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:02.235+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:02.254 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:02.342 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:02.338+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:02.418 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:02.419 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:02.433 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:02.443 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:02.438+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:02.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:02.544 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:02.539+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:02.557 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:02.557 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:02.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:02.657 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:02.655+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:02.674 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:02.674 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:02.690 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:02.706 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:02.722 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:02.722 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:02.734 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:02.734 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:02.744 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:02.744 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:02.745 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:02.742+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:02.751 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:02.768 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:02.768 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:02.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:02.787 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:02.810 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:02.810 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:02.871 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:02.870+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.148280346000007, 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:35:02.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:35:02.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.147902354999999, 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:35:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:02.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:35:02.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:35:02.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.936 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.147683099999995, 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:35:02.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.940 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.147285937000007, 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.941 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:35:02.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.944 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.146882149000007, 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:35:02.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:35:02.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:35:02.948 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.146426387000005, 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:35:02.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:35:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:35:02.952 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95552 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8680448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.139199841000007, 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104735093, 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95552 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.953 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786610+0000", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786227+0000", 2024-09-21T15:35:02.954 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996171 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786614+0000", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00038691499999999999 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786622+0000", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3809999999999999e-06 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786638+0000", 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5274e-05 2024-09-21T15:35:02.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891007+0000", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104369038 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891162+0000", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015558299999999999 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891345+0000", 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000182976 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.956 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.138833552999998, 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.957 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.958 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.959 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:35:02.960 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.138121709999993, 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:35:02.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:35:02.962 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:35:02.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:35:02.964 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.134861162999997, 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:35:02.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:35:02.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:35:02.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95565 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9737216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.134420943999999, 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474641799999999, 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95565 2024-09-21T15:35:02.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791389+0000", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790981+0000", 2024-09-21T15:35:02.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995918 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791397+0000", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041563499999999997 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791412+0000", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5455999999999999e-05 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791420+0000", 2024-09-21T15:35:02.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6149999999999999e-06 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895839+0000", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10441916900000001 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895973+0000", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000134571 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896136+0000", 2024-09-21T15:35:02.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016216000000000001 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.134109520999999, 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.973 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.109181913, 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.977 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:02.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:35:02.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.108552256999999, 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:35:02.981 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.985 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.108147794000004, 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:02.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:35:02.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:35:02.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.989 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.107609401000005, 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:02.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:35:02.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:35:02.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.993 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.107082383000005, 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:02.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:35:02.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.106660657999996, 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:02.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:02.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:03.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:03.001 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.106245255999994, 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:35:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:03.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 68.105694829000001, 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:03.006 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:03.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:03.010 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:03.010 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:35:03.011 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:03.012 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:03.010+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:03.112 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:03.107+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:03.148 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:03.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:03.148 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:03.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:03.214 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:03.211+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:03.277 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.277 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:03.277 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:03.277 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:03.277 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:03.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:03.335 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:03.334+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:03.415 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:03.410+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:03.420 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.420 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:03.420 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187874 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868439716000001, 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089103661000000001, 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187874 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.421 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550069+0000", 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550082+0000", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3054000000000001e-05 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550099+0000", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7122999999999999e-05 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550113+0000", 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3526999999999999e-05 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.423 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.638958+0000", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088845460000000001 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639075+0000", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117033 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639174+0000", 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8381000000000006e-05 2024-09-21T15:35:03.424 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187875 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868384898999999, 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089423839000000005, 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.425 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187875 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550124+0000", 2024-09-21T15:35:03.426 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550131+0000", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7299999999999999e-06 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550148+0000", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6909e-05 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550164+0000", 2024-09-21T15:35:03.427 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6010000000000001e-05 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639254+0000", 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089090091999999996 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639387+0000", 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013300900000000001 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.428 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639549+0000", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016201900000000001 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187876 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868334034, 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089634236000000006, 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187876 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.430 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550174+0000", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550181+0000", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1870000000000002e-06 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550197+0000", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6617e-05 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550213+0000", 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5566e-05 2024-09-21T15:35:03.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639619+0000", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089406157 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639711+0000", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2303e-05 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639810+0000", 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8765999999999996e-05 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.432 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187877 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868285044, 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089883745000000001, 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187877 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.434 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550224+0000", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550230+0000", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550247+0000", 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7445999999999999e-05 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.435 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550261+0000", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.314e-05 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639864+0000", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089603631000000003 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639969+0000", 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010485200000000001 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640109+0000", 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013950200000000001 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187878 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868236937999999, 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090091083000000002, 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.437 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187878 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550272+0000", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550278+0000", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550295+0000", 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7037e-05 2024-09-21T15:35:03.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550308+0000", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.306e-05 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640175+0000", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089867531 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640269+0000", 2024-09-21T15:35:03.440 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3774e-05 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640364+0000", 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4840999999999995e-05 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187879 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868190904999999, 2024-09-21T15:35:03.441 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090330875000000005, 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187879 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.442 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550318+0000", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550324+0000", 2024-09-21T15:35:03.443 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1929999999999998e-06 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550341+0000", 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7180000000000002e-05 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550354+0000", 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3074000000000001e-05 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.444 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640420+0000", 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090065509000000002 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640517+0000", 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6803000000000001e-05 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640650+0000", 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013306900000000001 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187880 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868143331999999, 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090571654000000001, 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187880 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550366+0000", 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550371+0000", 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8939999999999998e-06 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550388+0000", 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6745999999999999e-05 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.448 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550402+0000", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.329e-05 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640716+0000", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090314426000000003 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640813+0000", 2024-09-21T15:35:03.449 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7497000000000005e-05 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640938+0000", 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124771 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187881 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868097246000001, 2024-09-21T15:35:03.450 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090817446999999996, 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187881 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.451 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550412+0000", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550418+0000", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2929999999999997e-06 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.452 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550435+0000", 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7073000000000001e-05 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550448+0000", 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640992+0000", 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090544168999999994 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.453 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641103+0000", 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110806 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641230+0000", 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012683200000000001 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187882 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.454 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.868049782, 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090998446999999996, 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187882 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550459+0000", 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.456 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550465+0000", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8560000000000003e-06 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550482+0000", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6702999999999999e-05 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550494+0000", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2887e-05 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.457 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641273+0000", 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090778677000000002 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641344+0000", 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1348999999999997e-05 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641459+0000", 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114135 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.458 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187883 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.867982276999999, 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091270417000000006, 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187883 2024-09-21T15:35:03.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.460 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550526+0000", 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550533+0000", 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1069999999999999e-06 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550552+0000", 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8646000000000001e-05 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.461 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550568+0000", 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5967000000000001e-05 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641512+0000", 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090943916 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.462 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641657+0000", 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014500900000000001 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641798+0000", 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014126500000000001 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187884 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.463 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.867929589999999, 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091471644000000005, 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187884 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.464 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.465 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550579+0000", 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550585+0000", 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2029999999999997e-06 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.466 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550603+0000", 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7430000000000001e-05 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550616+0000", 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3623e-05 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641866+0000", 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091249446999999997 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.467 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641959+0000", 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3412999999999995e-05 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642052+0000", 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2731000000000004e-05 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.468 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187885 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.867365455000002, 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091213787000000005, 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187885 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.469 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.470 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550627+0000", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9994826 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551154+0000", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052656199999999995 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551178+0000", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4273e-05 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551195+0000", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7499999999999998e-05 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642107+0000", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090911480000000003 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642219+0000", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112209 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642358+0000", 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013906199999999999 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187886 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.867298972, 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091413275000000002, 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.473 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187886 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551210+0000", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551277+0000", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7926e-05 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551299+0000", 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1759999999999998e-05 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551315+0000", 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5489e-05 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642432+0000", 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091117087999999999 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642523+0000", 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1644999999999994e-05 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642624+0000", 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100777 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187887 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.865608786999999, 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089972300000000005, 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.478 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187887 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.479 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551327+0000", 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9984255 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552910+0000", 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015832330000000001 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552937+0000", 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7183e-05 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552955+0000", 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7629e-05 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642660+0000", 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089705650999999997 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642776+0000", 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116109 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642873+0000", 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7010999999999997e-05 2024-09-21T15:35:03.482 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187888 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 593920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.865538913999998, 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090280418000000001, 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.483 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187888 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:03.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552970+0000", 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552977+0000", 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4460000000000004e-06 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552995+0000", 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8386999999999999e-05 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553009+0000", 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3597e-05 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642950+0000", 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089941260999999995 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643087+0000", 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013637499999999999 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643251+0000", 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016482199999999999 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.487 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187889 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 618496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.865489554, 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090483776000000002, 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187889 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553019+0000", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553026+0000", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1229999999999997e-06 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553072+0000", 2024-09-21T15:35:03.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6588999999999997e-05 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553088+0000", 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.558e-05 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643322+0000", 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090234272000000004 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643411+0000", 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9073999999999995e-05 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643504+0000", 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3084999999999994e-05 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187890 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 643072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.865404478999999, 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090643425, 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.493 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187890 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553104+0000", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553111+0000", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553130+0000", 2024-09-21T15:35:03.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.836e-05 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553147+0000", 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7750000000000001e-05 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643540+0000", 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090392792999999999 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643655+0000", 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011505700000000001 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643749+0000", 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3638e-05 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.497 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187900 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 888832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.861024733000001, 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089230480000000001, 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187900 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557484+0000", 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557490+0000", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557508+0000", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7363999999999998e-05 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557521+0000", 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.362e-05 2024-09-21T15:35:03.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646492+0000", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088970448999999993 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646603+0000", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011123199999999999 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646716+0000", 2024-09-21T15:35:03.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112759 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187901 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.860977269999999, 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089469512000000001, 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187901 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557532+0000", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557537+0000", 2024-09-21T15:35:03.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7230000000000001e-06 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557555+0000", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7547e-05 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557568+0000", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2945999999999999e-05 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646770+0000", 2024-09-21T15:35:03.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089201761000000004 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646869+0000", 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9148999999999995e-05 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647002+0000", 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013329600000000001 2024-09-21T15:35:03.506 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187902 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.860929949999999, 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089715839000000006, 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.507 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187902 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557579+0000", 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557585+0000", 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.913e-06 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557602+0000", 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7603000000000001e-05 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557616+0000", 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3713000000000001e-05 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647074+0000", 2024-09-21T15:35:03.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089457866999999996 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647182+0000", 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107559 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647296+0000", 2024-09-21T15:35:03.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011416099999999999 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.512 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:03.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:03.516 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:03.511+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:03.617 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:03.615+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:03.682 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.682 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:03.682 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:03.682 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:03.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:03.739 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:03.738+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:35:03.819 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:03.815+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:03.833 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.833 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:03.833 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:03.833 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:03.833 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:03.848 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:03.919 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:03.918+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.967 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 108.842548154, 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:03.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:35:03.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:35:03.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.973 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.974 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.400391940999995, 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:03.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:35:03.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:35:03.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.400367195000001, 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:35:03.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.400322244999998, 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396465586999994, 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.987 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:35:03.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.990 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396421227000005, 2024-09-21T15:35:03.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:03.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:35:03.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:03.995 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396367878000007, 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:03.999 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396336485000006, 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396290758999996, 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.007 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396261386000006, 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:04.008 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:35:04.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:35:04.010 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.011 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396236826000006, 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:04.012 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:35:04.013 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.014 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.396210189000001, 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.016 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.017 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.018 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392371621999999, 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:35:04.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:35:04.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392341802000004, 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:35:04.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:35:04.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:35:04.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392316272000002, 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:35:04.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:35:04.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.031 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392290566, 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392240525999995, 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:35:04.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.040 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392205726, 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:35:04.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:35:04.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.392181077000004, 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:35:04.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:35:04.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:35:04.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:35:04.048 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.388130669999995, 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.053 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:04.055 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:04.055 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:04.050+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:04.156 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:04.154+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:04.224 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:04.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:04.225 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:04.239 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:04.257 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:04.255+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:04.358 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:04.359+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:04.401 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:04.401 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:04.402 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:04.402 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:04.402 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:04.416 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:04.460 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:04.455+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:04.536 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 43.219236629999997, 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.537 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.301316550999999, 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:04.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.301282276999999, 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:04.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:04.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.301272016999999, 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.551 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:04.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:04.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297398922999999, 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:04.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:04.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297390154999999, 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:04.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297381469000001, 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:04.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297373021999999, 2024-09-21T15:35:04.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:04.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.571 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297364480999999, 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:04.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:04.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:04.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297355749000001, 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297346814000001, 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.297298175000002, 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293417414, 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:04.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.592 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293408575000001, 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293399714, 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293390994999999, 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293382510000001, 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293373773999999, 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:04.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.612 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293365250000001, 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.613 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:04.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:04.616 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 18.293356798000001, 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:04.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:04.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:04.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:04.620 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:04.621 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:04.622 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:04.622 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:04.618+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:04.723 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:04.722+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:04.798 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:04.798 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:04.798 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:04.798 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:04.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:04.824 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:04.819+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:04.829 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:04.925 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:04.923+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:04.984 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:04.984 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:04.984 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:05.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:05.002 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:05.003 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:05.018 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:05.018 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:05.026 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:05.023+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:05.034 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:05.034 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:05.050 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:05.051 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:05.070 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:05.071 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:05.091 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:05.091 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:05.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:05.111 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:05.116 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.117 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.127 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:05.123+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:05.131 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:05.132 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:05.132 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:05.228 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:05.226+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:05.264 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.264 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:05.329 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:05.326+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:05.402 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.402 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:05.430 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:05.426+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:05.531 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:05.526+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:05.535 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.535 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:05.631 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:05.630+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:05.667 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.667 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:05.731 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:05.727+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:05.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:35:05.807 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.807 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:05.824 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:05.832 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:05.830+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:05.933 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:05.931+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:05.998 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:05.998 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.012 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:06.061 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:06.059+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:06.134 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:06.131+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:06.181 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:06.182 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.195 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:06.235 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:06.230+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:06.336 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:06.331+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:06.360 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:06.360 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:06.436 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:06.434+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:06.537 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:06.535+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:06.537 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:06.537 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.552 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:06.637 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:06.634+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:06.720 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:06.720 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.734 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:35:06.781 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:06.779+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:06.783 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:06.783 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:06.783 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-21T15:35:06.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-21T15:35:06.839 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:06.835+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:06.878 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:06.878 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:06.895 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:06.939 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:06.938+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:07.041 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:07.039+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:07.057 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:07.057 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:07.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:07.103 INFO:tasks.ceph.mon.a.smithi107.stderr:2024-09-21T15:35:07.098+0000 7fbc162d8640 -1 mon.a@0(leader).osd e125 definitely_dead 0 2024-09-21T15:35:07.142 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:07.138+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:07.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:07.195 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:07.195 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:07.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:07.243 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:07.239+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:07.289 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:07.305 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:07.305 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:07.323 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:07.339 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:07.339 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:07.343 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:07.342+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:07.359 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:07.360 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:07.383 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:07.383 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:07.383 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:07.383 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:07.383 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:07.394 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:07.400 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:07.415 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:07.415 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:07.436 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:07.436 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:07.444 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:07.442+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187874 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:07.533 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981927449000001, 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089103661000000001, 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187874 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550070+0000", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550069+0000", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550082+0000", 2024-09-21T15:35:07.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3054000000000001e-05 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550099+0000", 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7122999999999999e-05 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550113+0000", 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3526999999999999e-05 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.638958+0000", 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088845460000000001 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639075+0000", 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117033 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639174+0000", 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8381000000000006e-05 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187875 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981872631999998, 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089423839000000005, 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187875 2024-09-21T15:35:07.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550125+0000", 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550124+0000", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550131+0000", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7299999999999999e-06 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550148+0000", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6909e-05 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550164+0000", 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6010000000000001e-05 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639254+0000", 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089090091999999996 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639387+0000", 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013300900000000001 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639549+0000", 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016201900000000001 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187876 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981821767, 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089634236000000006, 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187876 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550176+0000", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550174+0000", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550181+0000", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1870000000000002e-06 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550197+0000", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6617e-05 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550213+0000", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5566e-05 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639619+0000", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089406157 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639711+0000", 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2303e-05 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639810+0000", 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8765999999999996e-05 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187877 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981772777, 2024-09-21T15:35:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089883745000000001, 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187877 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550225+0000", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550224+0000", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550230+0000", 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1240000000000002e-06 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550247+0000", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7445999999999999e-05 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550261+0000", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.314e-05 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639864+0000", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089603631000000003 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.639969+0000", 2024-09-21T15:35:07.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010485200000000001 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640109+0000", 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013950200000000001 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187878 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 348160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981724670999998, 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090091083000000002, 2024-09-21T15:35:07.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187878 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550273+0000", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550272+0000", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550278+0000", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550295+0000", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7037e-05 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550308+0000", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.306e-05 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640175+0000", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089867531 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640269+0000", 2024-09-21T15:35:07.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3774e-05 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640364+0000", 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4840999999999995e-05 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187879 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:07.554 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981678637999998, 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090330875000000005, 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187879 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550319+0000", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550318+0000", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550324+0000", 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1929999999999998e-06 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550341+0000", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7180000000000002e-05 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550354+0000", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3074000000000001e-05 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640420+0000", 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090065509000000002 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640517+0000", 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6803000000000001e-05 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640650+0000", 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013306900000000001 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187880 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 397312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.558 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981631064999998, 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090571654000000001, 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187880 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550367+0000", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550366+0000", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550371+0000", 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8939999999999998e-06 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550388+0000", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6745999999999999e-05 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550402+0000", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.329e-05 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640716+0000", 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090314426000000003 2024-09-21T15:35:07.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640813+0000", 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7497000000000005e-05 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640938+0000", 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000124771 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.562 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187881 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 421888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981584979000001, 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090817446999999996, 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187881 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550413+0000", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550412+0000", 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550418+0000", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2929999999999997e-06 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550435+0000", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7073000000000001e-05 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550448+0000", 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.640992+0000", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090544168999999994 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641103+0000", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110806 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641230+0000", 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012683200000000001 2024-09-21T15:35:07.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187882 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981537514999999, 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090998446999999996, 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187882 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550460+0000", 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550459+0000", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550465+0000", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.8560000000000003e-06 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550482+0000", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6702999999999999e-05 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550494+0000", 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2887e-05 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641273+0000", 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090778677000000002 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641344+0000", 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1348999999999997e-05 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641459+0000", 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114135 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187883 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 471040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981470009999999, 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091270417000000006, 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.571 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187883 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550528+0000", 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550526+0000", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550533+0000", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1069999999999999e-06 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550552+0000", 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8646000000000001e-05 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550568+0000", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5967000000000001e-05 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641512+0000", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090943916 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641657+0000", 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014500900000000001 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641798+0000", 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014126500000000001 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187884 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.981417322999999, 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091471644000000005, 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187884 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550580+0000", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550579+0000", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550585+0000", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2029999999999997e-06 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550603+0000", 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7430000000000001e-05 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550616+0000", 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3623e-05 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641866+0000", 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091249446999999997 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.641959+0000", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3412999999999995e-05 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642052+0000", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2731000000000004e-05 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187885 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.980853188000001, 2024-09-21T15:35:07.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091213787000000005, 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187885 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551144+0000", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.550627+0000", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9994826 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551154+0000", 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052656199999999995 2024-09-21T15:35:07.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551178+0000", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4273e-05 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551195+0000", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7499999999999998e-05 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642107+0000", 2024-09-21T15:35:07.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090911480000000003 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642219+0000", 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112209 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642358+0000", 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013906199999999999 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.583 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187886 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 544768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.980786705, 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091413275000000002, 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187886 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551211+0000", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551210+0000", 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551277+0000", 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7926e-05 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551299+0000", 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1759999999999998e-05 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551315+0000", 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5489e-05 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642432+0000", 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091117087999999999 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642523+0000", 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1644999999999994e-05 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642624+0000", 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100777 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187887 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 569344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.979096519999999, 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089972300000000005, 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.588 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187887 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552901+0000", 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.551327+0000", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9984255 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552910+0000", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0015832330000000001 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552937+0000", 2024-09-21T15:35:07.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7183e-05 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552955+0000", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7629e-05 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642660+0000", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089705650999999997 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642776+0000", 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000116109 2024-09-21T15:35:07.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642873+0000", 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7010999999999997e-05 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187888 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 593920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.979026647000001, 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090280418000000001, 2024-09-21T15:35:07.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187888 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552971+0000", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552970+0000", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552977+0000", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4460000000000004e-06 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.552995+0000", 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8386999999999999e-05 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553009+0000", 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3597e-05 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.642950+0000", 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089941260999999995 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643087+0000", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013637499999999999 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643251+0000", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00016482199999999999 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187889 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 618496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:07.596 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.978977286999999, 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090483776000000002, 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187889 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553020+0000", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553019+0000", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553026+0000", 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.1229999999999997e-06 2024-09-21T15:35:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553072+0000", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6588999999999997e-05 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553088+0000", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.558e-05 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643322+0000", 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090234272000000004 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643411+0000", 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9073999999999995e-05 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643504+0000", 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3084999999999994e-05 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.600 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187890 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 643072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.978892212000002, 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090643425, 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187890 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553105+0000", 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553104+0000", 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553111+0000", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553130+0000", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.836e-05 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.553147+0000", 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7750000000000001e-05 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643540+0000", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090392792999999999 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643655+0000", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011505700000000001 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.643749+0000", 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3638e-05 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.604 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187900 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 888832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.974512466, 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089230480000000001, 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187900 2024-09-21T15:35:07.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557485+0000", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557484+0000", 2024-09-21T15:35:07.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557490+0000", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557508+0000", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7363999999999998e-05 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557521+0000", 2024-09-21T15:35:07.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.362e-05 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646492+0000", 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.088970448999999993 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646603+0000", 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011123199999999999 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646716+0000", 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000112759 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187901 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 913408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.974465002999999, 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089469512000000001, 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.609 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187901 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557533+0000", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557532+0000", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557537+0000", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7230000000000001e-06 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557555+0000", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7547e-05 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557568+0000", 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2945999999999999e-05 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646770+0000", 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089201761000000004 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.646869+0000", 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9148999999999995e-05 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647002+0000", 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013329600000000001 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:187902 3.b 3:d40d7cc2:::rbd_data.1191eefab886.0000000000000006:2a [read 937984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.974417682999999, 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089715839000000006, 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:07.613 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 187902 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557580+0000", 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557579+0000", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557585+0000", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.913e-06 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557602+0000", 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7603000000000001e-05 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.557616+0000", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3713000000000001e-05 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647074+0000", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089457866999999996 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647182+0000", 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107559 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:07.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:44.647296+0000", 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011416099999999999 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:07.617 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:07.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:07.619 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:07.615+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:07.719 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:07.715+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:07.745 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:07.745 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:07.745 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:07.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:07.769 INFO:teuthology.orchestra.run.smithi107.stderr:marked out osd.0. 2024-09-21T15:35:07.819 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:07.815+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:07.882 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:07.882 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:07.882 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:07.882 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:07.882 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:07.896 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:07.943 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:07.942+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:35:08.018 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 112.89320175, 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.021 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:35:08.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:35:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:08.024 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.025 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.451045536999999, 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:08.026 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.027 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.028 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.451020791000005, 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.030 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.032 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.033 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.450975841000002, 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.034 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:08.035 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:35:08.036 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.037 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.447119182999998, 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.040 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.041 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.042 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.447074822999994, 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:35:08.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:35:08.045 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.447021473999996, 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.047 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.048 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.049 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:35:08.050 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.446990080999996, 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.052 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:35:08.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.446944354999999, 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:35:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:35:08.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.446914981999996, 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:08.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:35:08.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.446890421999996, 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.064 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:08.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.446863785000005, 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:35:08.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.443025218000002, 2024-09-21T15:35:08.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:35:08.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442995397999994, 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442969868000006, 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:35:08.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.085 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442944162000003, 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:08.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442894121999998, 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.093 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442859322000004, 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.094 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.442834672999993, 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:35:08.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.438784265999999, 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:35:08.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:35:08.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.107 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:08.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:08.109 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:08.106+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:08.209 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:08.206+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:08.279 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:08.280 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:08.280 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:08.280 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:08.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:08.310 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:08.307+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:08.411 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:08.410+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:08.453 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:08.453 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:08.453 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:08.453 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:08.453 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:08.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:08.512 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:08.511+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:08.612 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:08.607+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:08.633 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 47.317244266000003, 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:08.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:08.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:08.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:08.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:08.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:08.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:08.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.399324187000001, 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.646 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.399289913, 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:08.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.399279653000001, 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:08.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:08.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395406559000001, 2024-09-21T15:35:08.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:08.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395397791000001, 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:08.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:08.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395389105, 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:08.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:08.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:08.668 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395380658000001, 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:08.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395372117000001, 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:08.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395363385, 2024-09-21T15:35:08.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:08.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.681 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395354449999999, 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.395305811, 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:08.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:08.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:08.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391425049999999, 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:08.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391416210999999, 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.695 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391407350000001, 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:08.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391398631000001, 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:08.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391390145999999, 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:08.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391381410000001, 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.716 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391372885999999, 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:08.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 22.391364434, 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:08.721 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:08.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:08.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:08.725 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:08.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:08.727 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:08.723+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:08.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:35:08.839 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:08.838+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:08.859 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:08.859 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:08.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:08.960 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:08.958+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:09.028 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:09.028 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:09.029 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:09.029 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:09.029 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:09.029 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:09.026+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:09.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:09.130 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:09.127+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:09.213 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:09.213 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:09.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:09.213 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:09.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.84319266599999, 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:09.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.219 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.81498149000001, 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:09.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:09.223 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 135.625682122, 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:09.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 135.49429743100001, 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.229 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:09.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 132.333200027, 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:09.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 132.33314748000001, 2024-09-21T15:35:09.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:09.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:09.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:09.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:09.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 132.333099355, 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:09.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 132.32070246699999, 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:09.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:09.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.75992948499999, 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:09.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:09.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:09.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.69290511899999, 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.267 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.55778498800001, 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.51712284499999, 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.273 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.51710962800001, 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:09.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.517096821, 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.517056414, 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.286 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:09.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:09.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:09.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.517031946, 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.290 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.497809958, 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:09.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:09.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 130.41050637500001, 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:09.300 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:09.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:09.305 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 62.415820932000003, 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:09.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:09.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 62.001561584999997, 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:09.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:09.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:09.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:09.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:09.314 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:09.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:09.315 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:09.314+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:09.416 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:09.411+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:09.447 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:09.447 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:09.447 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:09.448 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:09.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:09.461 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:09.549 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:09.547+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:09.607 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:09.607 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:09.617 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:09.615+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:09.623 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:09.624 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:09.625 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:09.638 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:09.639 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:09.640 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:09.655 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:09.655 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:09.673 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:09.673 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:09.690 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:09.690 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:09.707 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:09.718 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:09.714+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:09.725 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:09.725 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:09.784 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:35:09.819 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:09.815+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:09.830 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:09.830 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:09.849 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:09.941 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:09.939+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:10.020 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:10.018+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:10.044 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.044 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:10.121 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:10.118+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:10.222 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:10.218+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:10.249 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.250 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:10.323 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:10.318+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:10.389 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.389 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.405 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:10.449 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:10.447+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:10.524 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:10.522+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:10.529 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.529 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:10.629 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:10.627+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:10.708 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.708 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:10.765 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:10.763+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:10.827 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:10.822+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:10.896 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:10.896 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:10.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:10.927 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:10.926+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:11.028 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:11.026+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:11.035 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.035 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.049 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:11.129 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:11.126+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:11.167 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.167 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.181 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:11.230 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:11.226+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:11.302 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.302 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:11.364 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:11.362+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:11.432 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:11.427+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:11.449 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.449 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:11.533 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:11.531+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:11.628 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.629 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.634 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:11.631+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:11.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:11.735 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:11.730+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:11.742 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:11.765 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:11.765 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:11.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:11.819 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:35:11.836 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:11.831+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:11.922 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:11.922 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:11.936 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:11.931+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:11.941 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:11.955 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:11.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:11.955 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:11.955 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:11.961 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:11.961 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:11.972 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:11.986 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:12.020 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:12.036 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:12.034+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:12.040 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:12.060 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:12.060 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:12.080 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:12.080 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:12.087 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:12.087 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:12.087 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:12.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:12.152 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:12.150+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:12.239 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:12.235+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 117.14211254600001, 2024-09-21T15:35:12.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:35:12.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:35:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:12.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:12.274 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.699956333000003, 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:35:12.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.699931586999995, 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:35:12.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.699886637000006, 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.283 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:35:12.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.287 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.696029979000002, 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:12.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:35:12.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695985618999998, 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:35:12.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:35:12.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.69593227, 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:35:12.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:35:12.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695900877, 2024-09-21T15:35:12.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.304 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695855151000004, 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:12.305 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.306 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.307 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.308 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695825778, 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:35:12.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.310 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.311 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.312 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695801218, 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.314 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.315 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.317 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.695774580999995, 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:12.318 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:35:12.319 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:35:12.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.321 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691936014000007, 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.322 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.323 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.324 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691906193999998, 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:35:12.327 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:35:12.328 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:35:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691880663999996, 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.330 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:35:12.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:35:12.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691854957999993, 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:35:12.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.335 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:35:12.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:35:12.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691804918000003, 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.339 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:35:12.341 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.342 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691770117999994, 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.343 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.345 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.691745468999997, 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.348 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:35:12.349 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.350 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:12.351 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 92.687695062000003, 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.352 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.353 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.354 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.355 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:12.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:12.357 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:12.355+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:12.458 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:12.455+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:12.480 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:12.480 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:12.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:12.480 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:12.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:12.558 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:12.554+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:12.609 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:12.609 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:12.609 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:12.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:12.659 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:12.655+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:12.759 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:12.758+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:12.797 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:12.797 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:12.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 51.480026328999998, 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:12.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:12.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.562106249999999, 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:12.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.562071975999999, 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.562061715999999, 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.813 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:12.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:12.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558188621999999, 2024-09-21T15:35:12.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.821 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558179853999999, 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.822 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:12.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558171168000001, 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.829 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558162720999999, 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:12.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.833 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558154179999999, 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:12.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:12.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:12.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558145448000001, 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558136513000001, 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:12.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:12.845 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.558087874000002, 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:12.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:12.847 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:12.848 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:12.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554207113, 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:12.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.852 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554198274000001, 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.855 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.856 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:12.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554189413, 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.860 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.861 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:12.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554180693999999, 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:12.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.864 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.865 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:12.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.867 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554172209000001, 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:12.868 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:12.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:12.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.871 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554163472999999, 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.872 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.874 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554154949000001, 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.876 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.877 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.878 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.879 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 26.554146497000001, 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:12.880 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:12.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:12.882 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.883 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:12.884 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:12.885 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:12.885 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:12.885 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T15:35:12.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T15:35:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:35:12.887 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:12.888 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:12.882+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:12.987 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:12.986+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:13.053 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:13.053 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:13.053 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:13.053 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:13.068 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:13.112 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:13.110+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:13.190 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:13.187+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:13.226 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:13.226 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:13.226 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 131, 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "fsid": "09582c7a-ec34-4064-810b-b65e59e21b91", 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "created": "2024-09-21T15:32:16.991419+0000", 2024-09-21T15:35:13.231 INFO:teuthology.orchestra.run.smithi107.stdout: "modified": "2024-09-21T15:35:12.796563+0000", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "last_up_change": "2024-09-21T15:32:28.438444+0000", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "last_in_change": "2024-09-21T15:35:07.759857+0000", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_num": 5806080, 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_set": [ 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "nodeep-scrub", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "noscrub", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "pglog_hardlimit", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "purged_snapdirs", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_deletes", 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "sortbitwise" 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_version": 4, 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T15:35:13.232 INFO:teuthology.orchestra.run.smithi107.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_snapshot": "", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_max": 3, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd": 8, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "require_min_compat_client": "luminous", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min_compat_client": "luminous", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "require_osd_release": "reef", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_crimson": false, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "pools": [ 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 1, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": ".mgr", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:29.715035+0000", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool", 2024-09-21T15:35:13.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 1, 2024-09-21T15:35:13.234 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "16", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:13.235 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:13.236 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:13.237 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "options": { 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_max": 32, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_min": 1 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": {} 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 6.9000000953674316, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 6.9000000953674316, 2024-09-21T15:35:13.238 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 0.28999999165534973, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 2, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 2, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 2, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "rbd", 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:35.528036+0000", 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:13.239 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 8, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 8, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 8, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 8, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 8, 2024-09-21T15:35:13.240 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "20", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 3, 2024-09-21T15:35:13.241 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 20, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:13.242 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:13.243 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:35:13.244 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": "pool_client.0", 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "create_time": "2024-09-21T15:32:44.899492+0000", 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 8193, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": 1, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 2, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": 1, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_rule": 0, 2024-09-21T15:35:13.245 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_count": 0, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_target": 0, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "object_hash": 2, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscale_mode": "off", 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": 32, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num": 32, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_placement_num_target": 32, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_target": 32, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num_pending": 32, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "last_pg_merge_meta": { 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "source_pgid": "0.0", 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "ready_epoch": 0, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_started": 0, 2024-09-21T15:35:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: "last_epoch_clean": 0, 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "source_version": "0'0", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "target_version": "0'0" 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "last_change": "131", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend": "0", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "auid": 0, 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_mode": "selfmanaged", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_seq": 59, 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_epoch": 131, 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_snaps": [], 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps": "[]", 2024-09-21T15:35:13.247 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_bytes": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "quota_max_objects": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tiers": [], 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tier_of": -1, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "read_tier": -1, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "write_tier": -1, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_mode": "none", 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_bytes": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "target_max_objects": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_flush_age": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "cache_min_evict_age": 0, 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profile": "", 2024-09-21T15:35:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_params": { 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "none" 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_period": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_count": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "use_gmt_hitset": true, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min_read_recency_for_promote": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min_write_recency_for_promote": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "hit_set_search_last_n": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "grade_table": [], 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "stripe_width": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "expected_num_objects": 0, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "fast_read": false, 2024-09-21T15:35:13.249 INFO:teuthology.orchestra.run.smithi107.stdout: "options": {}, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "application_metadata": { 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd": {} 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "read_balance": { 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "score_acting": 1.75, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "score_stable": 1.75, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "optimal_score": 1, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_acting": 1.75, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "raw_score_stable": 1.75, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity": 1, 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "d2f9d147-bbb0-470e-8441-d891316be3ee", 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 0, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 0, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 128, 2024-09-21T15:35:13.251 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6824", 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6825", 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6826", 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6827", 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.253 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6830", 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6831", 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.254 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6828", 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6829", 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 379929313 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6825/379929313", 2024-09-21T15:35:13.255 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6827/379929313", 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6831/379929313", 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6829/379929313", 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "e13f399d-5ddd-47e1-879e-eda2f7ec930f", 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.256 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 129, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6816", 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6817", 2024-09-21T15:35:13.257 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6818", 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6819", 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.258 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6822", 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6823", 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.259 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6820", 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6821", 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 615179745 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6817/615179745", 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6819/615179745", 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6823/615179745", 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6821/615179745", 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.261 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "84d52d47-67ab-43ca-a983-5568d9cf1975", 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 89, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6802", 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6803", 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.264 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6806", 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6807", 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.265 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6804", 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6805", 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3712995483 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6801/3712995483", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6803/3712995483", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6807/3712995483", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6805/3712995483", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "adbb6235-d8c3-4b20-a7e4-0046d22ed5fc", 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 0, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 100, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6808", 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6809", 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6810", 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6811", 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6814", 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6815", 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6812", 2024-09-21T15:35:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6813", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3212640731 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.107:6809/3212640731", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.107:6811/3212640731", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.107:6815/3212640731", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.107:6813/3212640731", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.272 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "f3780990-ed73-4f96-b93e-34e83e63d9e5", 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 129, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.273 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6800", 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6801", 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6802", 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6803", 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.275 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6806", 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6807", 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6804", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6805", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1739937222 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6801/1739937222", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6803/1739937222", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6807/1739937222", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6805/1739937222", 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.277 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "36f2dd75-f924-443d-ac44-02c8d0fa6cc6", 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 129, 2024-09-21T15:35:13.278 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6822", 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6824", 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.279 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6825", 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6827", 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.280 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6830", 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6831", 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.281 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6828", 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6829", 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4269648595 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6824/4269648595", 2024-09-21T15:35:13.282 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6827/4269648595", 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6831/4269648595", 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6829/4269648595", 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "cae6719a-cf33-472a-b127-964ed06f3c5c", 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 129, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6809", 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.284 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6810", 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6812", 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6813", 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6817", 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6819", 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6814", 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6816", 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3971003516 2024-09-21T15:35:13.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6810/3971003516", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6813/3971003516", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6819/3971003516", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6816/3971003516", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "uuid": "dde8b30b-cb83-4b0e-bdd9-b54ae839d666", 2024-09-21T15:35:13.288 INFO:teuthology.orchestra.run.smithi107.stdout: "up": 1, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "in": 1, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "weight": 1, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_affinity": 1, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_begin": 0, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "last_clean_end": 0, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "up_from": 13, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "up_thru": 129, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "down_at": 0, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "lost_at": 0, 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addrs": { 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6808", 2024-09-21T15:35:13.289 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6811", 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addrs": { 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.290 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6815", 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6818", 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addrs": { 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6823", 2024-09-21T15:35:13.291 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6826", 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addrs": { 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6820", 2024-09-21T15:35:13.292 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.170:6821", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3076528025 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "public_addr": "172.21.15.170:6811/3076528025", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_addr": "172.21.15.170:6818/3076528025", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_back_addr": "172.21.15.170:6826/3076528025", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "heartbeat_front_addr": "172.21.15.170:6821/3076528025", 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "state": [ 2024-09-21T15:35:13.293 INFO:teuthology.orchestra.run.smithi107.stdout: "exists", 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "up" 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_xinfo": [ 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 0, 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.294 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 65536, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.112615+0000", 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 1, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.099474+0000", 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.295 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 2, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.100534+0000", 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 3, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.296 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.993492+0000", 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 4, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:24.901492+0000", 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.297 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 5, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.197653+0000", 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 6, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.298 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.192998+0000", 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": 7, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "down_stamp": "0.000000", 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_probability": 0, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "laggy_interval": 0, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "features": 4540138322906710015, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "old_weight": 0, 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "last_purged_snaps_scrub": "2024-09-21T15:32:25.045812+0000", 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "dead_epoch": 0 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap": [ 2024-09-21T15:35:13.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "pgid": "3.14", 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "osds": [ 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: 2, 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: 6 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_items": [], 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_upmap_primaries": [], 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_temp": [], 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "primary_temp": [], 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklist": {}, 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "range_blocklist": {}, 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "erasure_code_profiles": { 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default": { 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-failure-domain": "osd", 2024-09-21T15:35:13.300 INFO:teuthology.orchestra.run.smithi107.stdout: "k": "2", 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "m": "1", 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "plugin": "jerasure", 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "technique": "reed_sol_van" 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "removed_snaps_queue": [ 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 30, 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 7 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "new_removed_snaps": [ 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "pool": 3, 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "snaps": [ 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "begin": 30, 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "length": 1 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "new_purged_snaps": [], 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_node_flags": {}, 2024-09-21T15:35:13.302 INFO:teuthology.orchestra.run.smithi107.stdout: "device_class_flags": {}, 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode": { 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_enabled": false, 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_bucket_count": 0, 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "degraded_stretch_mode": 0, 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "recovering_stretch_mode": 0, 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: "stretch_mode_bucket": 0 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.303 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:13.304 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:13.306 INFO:tasks.thrashosds.thrasher:j is {'epoch': 131, 'fsid': '09582c7a-ec34-4064-810b-b65e59e21b91', 'created': '2024-09-21T15:32:16.991419+0000', 'modified': '2024-09-21T15:35:12.796563+0000', 'last_up_change': '2024-09-21T15:32:28.438444+0000', 'last_in_change': '2024-09-21T15:35:07.759857+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T15:32:29.715035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T15:32:35.528036+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 20, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T15:32:44.899492+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 131, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'd2f9d147-bbb0-470e-8441-d891316be3ee', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6824', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6825', 'nonce': 379929313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6826', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6827', 'nonce': 379929313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6830', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6831', 'nonce': 379929313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6828', 'nonce': 379929313}, {'type': 'v1', 'addr': '172.21.15.107:6829', 'nonce': 379929313}]}, 'public_addr': '172.21.15.107:6825/379929313', 'cluster_addr': '172.21.15.107:6827/379929313', 'heartbeat_back_addr': '172.21.15.107:6831/379929313', 'heartbeat_front_addr': '172.21.15.107:6829/379929313', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e13f399d-5ddd-47e1-879e-eda2f7ec930f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 129, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6816', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6817', 'nonce': 615179745}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6818', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6819', 'nonce': 615179745}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6822', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6823', 'nonce': 615179745}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6820', 'nonce': 615179745}, {'type': 'v1', 'addr': '172.21.15.107:6821', 'nonce': 615179745}]}, 'public_addr': '172.21.15.107:6817/615179745', 'cluster_addr': '172.21.15.107:6819/615179745', 'heartbeat_back_addr': '172.21.15.107:6823/615179745', 'heartbeat_front_addr': '172.21.15.107:6821/615179745', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '84d52d47-67ab-43ca-a983-5568d9cf1975', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 89, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6800', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6801', 'nonce': 3712995483}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6802', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6803', 'nonce': 3712995483}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6806', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6807', 'nonce': 3712995483}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6804', 'nonce': 3712995483}, {'type': 'v1', 'addr': '172.21.15.107:6805', 'nonce': 3712995483}]}, 'public_addr': '172.21.15.107:6801/3712995483', 'cluster_addr': '172.21.15.107:6803/3712995483', 'heartbeat_back_addr': '172.21.15.107:6807/3712995483', 'heartbeat_front_addr': '172.21.15.107:6805/3712995483', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'adbb6235-d8c3-4b20-a7e4-0046d22ed5fc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6808', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6809', 'nonce': 3212640731}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6810', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6811', 'nonce': 3212640731}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6814', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6815', 'nonce': 3212640731}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6812', 'nonce': 3212640731}, {'type': 'v1', 'addr': '172.21.15.107:6813', 'nonce': 3212640731}]}, 'public_addr': '172.21.15.107:6809/3212640731', 'cluster_addr': '172.21.15.107:6811/3212640731', 'heartbeat_back_addr': '172.21.15.107:6815/3212640731', 'heartbeat_front_addr': '172.21.15.107:6813/3212640731', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f3780990-ed73-4f96-b93e-34e83e63d9e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 129, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6800', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6801', 'nonce': 1739937222}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6802', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6803', 'nonce': 1739937222}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6806', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6807', 'nonce': 1739937222}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6804', 'nonce': 1739937222}, {'type': 'v1', 'addr': '172.21.15.170:6805', 'nonce': 1739937222}]}, 'public_addr': '172.21.15.170:6801/1739937222', 'cluster_addr': '172.21.15.170:6803/1739937222', 'heartbeat_back_addr': '172.21.15.170:6807/1739937222', 'heartbeat_front_addr': '172.21.15.170:6805/1739937222', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '36f2dd75-f924-443d-ac44-02c8d0fa6cc6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 129, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6822', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6824', 'nonce': 4269648595}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6825', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6827', 'nonce': 4269648595}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6830', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6831', 'nonce': 4269648595}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6828', 'nonce': 4269648595}, {'type': 'v1', 'addr': '172.21.15.170:6829', 'nonce': 4269648595}]}, 'public_addr': '172.21.15.170:6824/4269648595', 'cluster_addr': '172.21.15.170:6827/4269648595', 'heartbeat_back_addr': '172.21.15.170:6831/4269648595', 'heartbeat_front_addr': '172.21.15.170:6829/4269648595', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'cae6719a-cf33-472a-b127-964ed06f3c5c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 129, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6809', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6810', 'nonce': 3971003516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6812', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6813', 'nonce': 3971003516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6817', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6819', 'nonce': 3971003516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6814', 'nonce': 3971003516}, {'type': 'v1', 'addr': '172.21.15.170:6816', 'nonce': 3971003516}]}, 'public_addr': '172.21.15.170:6810/3971003516', 'cluster_addr': '172.21.15.170:6813/3971003516', 'heartbeat_back_addr': '172.21.15.170:6819/3971003516', 'heartbeat_front_addr': '172.21.15.170:6816/3971003516', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'dde8b30b-cb83-4b0e-bdd9-b54ae839d666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 129, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6808', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6811', 'nonce': 3076528025}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6815', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6818', 'nonce': 3076528025}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6823', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6826', 'nonce': 3076528025}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6820', 'nonce': 3076528025}, {'type': 'v1', 'addr': '172.21.15.170:6821', 'nonce': 3076528025}]}, 'public_addr': '172.21.15.170:6811/3076528025', 'cluster_addr': '172.21.15.170:6818/3076528025', 'heartbeat_back_addr': '172.21.15.170:6826/3076528025', 'heartbeat_front_addr': '172.21.15.170:6821/3076528025', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.112615+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.099474+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.100534+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.993492+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:24.901492+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.197653+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.192998+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T15:32:25.045812+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.14', 'osds': [2, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T15:35:13.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:35:13.306 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:13.302+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:13.405 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:13.402+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 141.10182248199999, 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.470 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.471 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.472 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:13.473 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.474 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 141.073611306, 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:13.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:13.476 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:13.478 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 139.884311938, 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.479 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.481 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.482 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 139.752927247, 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.485 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:13.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.591829843, 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.488 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.489 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:13.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:13.491 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:13.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.493 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.591777296, 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:13.494 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.495 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.496 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.497 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.498 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.591729171, 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.499 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.500 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.502 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.503 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:13.504 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 136.57933228300001, 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:13.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:13.506 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:13.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:13.508 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 135.01855930100001, 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:13.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:13.512 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:13.513 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:13.514 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.95153493500001, 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:13.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:13.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:13.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:13.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.816414804, 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.77575266100001, 2024-09-21T15:35:13.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:13.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.532 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.77573944400001, 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.533 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.537 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.77572663699999, 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:13.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:13.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:13.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:13.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:13.543 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.77568622999999, 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:13.547 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.775661762, 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.756439774, 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:13.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 134.66913619100001, 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:13.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:13.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:13.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.562 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.674450747999998, 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:13.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:13.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:13.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 66.260191401, 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:13.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:13.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:13.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:13.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:13.571 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:13.572 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:13.572 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:13.571+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:13.652 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:35:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:35:13.670 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-21T15:35:12.809367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":581791623,"num_objects":1034,"num_object_clones":221,"num_object_copies":2068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1034,"num_whiteouts":424,"num_read":225689,"num_read_kb":472813,"num_write":15560,"num_write_kb":359473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":23936968,"num_keys_recovered":139,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12709,"ondisk_log_size":12709,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":409740,"kb_used_data":126892,"kb_used_omap":10,"kb_used_meta":282485,"kb_avail":660193140,"statfs":{"total":676457349120,"available":676037775360,"internally_reserved":0,"allocated":129937408,"data_stored":143428415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11149,"internal_metadata":289264755},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033837"},"pg_stats":[{"pgid":"3.1f","version":"125'634","reported_seq":5212,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.786908+0000","last_change":"2024-09-21T15:35:08.685138+0000","last_active":"2024-09-21T15:35:07.786908+0000","last_peered":"2024-09-21T15:35:06.871458+0000","last_clean":"2024-09-21T15:35:06.871458+0000","last_became_active":"2024-09-21T15:32:46.725835+0000","last_became_peered":"2024-09-21T15:32:46.725835+0000","last_unstale":"2024-09-21T15:35:07.786908+0000","last_undegraded":"2024-09-21T15:35:07.786908+0000","last_fullsized":"2024-09-21T15:35:07.786908+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:37.094981+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010613351,"stat_sum":{"num_bytes":19777423,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":6952,"num_read_kb":8279,"num_write":964,"num_write_kb":3270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"3.1e","version":"124'461","reported_seq":7199,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.778202+0000","last_change":"2024-09-21T15:35:07.778745+0000","last_active":"2024-09-21T15:35:07.778202+0000","last_peered":"2024-09-21T15:35:07.778202+0000","last_clean":"2024-09-21T15:35:07.778202+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:35:07.778202+0000","last_undegraded":"2024-09-21T15:35:07.778202+0000","last_fullsized":"2024-09-21T15:35:07.778202+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040214780000000002,"stat_sum":{"num_bytes":23958426,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":8765,"num_read_kb":21858,"num_write":575,"num_write_kb":26511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.1d","version":"124'152","reported_seq":541,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.787742+0000","last_change":"2024-09-21T15:35:07.788284+0000","last_active":"2024-09-21T15:35:07.787742+0000","last_peered":"2024-09-21T15:35:07.787742+0000","last_clean":"2024-09-21T15:35:07.787742+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:35:07.787742+0000","last_undegraded":"2024-09-21T15:35:07.787742+0000","last_fullsized":"2024-09-21T15:35:07.787742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052962410000000001,"stat_sum":{"num_bytes":16183579,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":283,"num_read_kb":6246,"num_write":202,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.1c","version":"125'277","reported_seq":5743,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.782419+0000","last_change":"2024-09-21T15:35:07.785874+0000","last_active":"2024-09-21T15:35:07.782419+0000","last_peered":"2024-09-21T15:35:07.782419+0000","last_clean":"2024-09-21T15:35:07.782419+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:35:07.782419+0000","last_undegraded":"2024-09-21T15:35:07.782419+0000","last_fullsized":"2024-09-21T15:35:07.782419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039406190000000002,"stat_sum":{"num_bytes":46730919,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":5541,"num_read_kb":22967,"num_write":293,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.1b","version":"125'368","reported_seq":9532,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.774467+0000","last_change":"2024-09-21T15:35:08.690438+0000","last_active":"2024-09-21T15:35:07.774467+0000","last_peered":"2024-09-21T15:35:06.771131+0000","last_clean":"2024-09-21T15:35:06.771131+0000","last_became_active":"2024-09-21T15:32:46.719044+0000","last_became_peered":"2024-09-21T15:32:46.719044+0000","last_unstale":"2024-09-21T15:35:07.774467+0000","last_undegraded":"2024-09-21T15:35:07.774467+0000","last_fullsized":"2024-09-21T15:35:07.774467+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:14.827226+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.016425298000000001,"stat_sum":{"num_bytes":12448712,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":10156,"num_read_kb":17275,"num_write":350,"num_write_kb":7281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"3.1a","version":"125'228","reported_seq":6956,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.772302+0000","last_change":"2024-09-21T15:35:07.773207+0000","last_active":"2024-09-21T15:35:07.772302+0000","last_peered":"2024-09-21T15:35:07.772302+0000","last_clean":"2024-09-21T15:35:07.772302+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:35:07.772302+0000","last_undegraded":"2024-09-21T15:35:07.772302+0000","last_fullsized":"2024-09-21T15:35:07.772302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001158039,"stat_sum":{"num_bytes":9969937,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":6674,"num_read_kb":35879,"num_write":221,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.19","version":"126'589","reported_seq":33376,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.785140+0000","last_change":"2024-09-21T15:35:07.785334+0000","last_active":"2024-09-21T15:35:07.785140+0000","last_peered":"2024-09-21T15:35:07.785140+0000","last_clean":"2024-09-21T15:35:07.785140+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:35:07.785140+0000","last_undegraded":"2024-09-21T15:35:07.785140+0000","last_fullsized":"2024-09-21T15:35:07.785140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010203073999999999,"stat_sum":{"num_bytes":42816880,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":34622,"num_read_kb":22737,"num_write":816,"num_write_kb":6873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.774066+0000","last_change":"2024-09-21T15:35:08.768678+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T15:35:07.774066+0000","last_undegraded":"2024-09-21T15:35:07.774066+0000","last_fullsized":"2024-09-21T15:35:07.774066+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:27:09.103120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"124'619","reported_seq":1906,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.782999+0000","last_change":"2024-09-21T15:35:08.685206+0000","last_active":"2024-09-21T15:35:07.782999+0000","last_peered":"2024-09-21T15:35:05.757999+0000","last_clean":"2024-09-21T15:35:05.757999+0000","last_became_active":"2024-09-21T15:32:46.725048+0000","last_became_peered":"2024-09-21T15:32:46.725048+0000","last_unstale":"2024-09-21T15:35:07.782999+0000","last_undegraded":"2024-09-21T15:35:07.782999+0000","last_fullsized":"2024-09-21T15:35:07.782999+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:40.643880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035760990000000001,"stat_sum":{"num_bytes":23580971,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":3894,"num_read_kb":11497,"num_write":925,"num_write_kb":7867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"3.16","version":"121'315","reported_seq":5865,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.785996+0000","last_change":"2024-09-21T15:35:07.787580+0000","last_active":"2024-09-21T15:35:07.785996+0000","last_peered":"2024-09-21T15:35:07.785996+0000","last_clean":"2024-09-21T15:35:07.785996+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:35:07.785996+0000","last_undegraded":"2024-09-21T15:35:07.785996+0000","last_fullsized":"2024-09-21T15:35:07.785996+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051062240000000004,"stat_sum":{"num_bytes":27601754,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":6967,"num_read_kb":11464,"num_write":316,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.15","version":"122'282","reported_seq":5113,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.796223+0000","last_change":"2024-09-21T15:35:07.797249+0000","last_active":"2024-09-21T15:35:07.796223+0000","last_peered":"2024-09-21T15:35:07.796223+0000","last_clean":"2024-09-21T15:35:07.796223+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:35:07.796223+0000","last_undegraded":"2024-09-21T15:35:07.796223+0000","last_fullsized":"2024-09-21T15:35:07.796223+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021200116000000001,"stat_sum":{"num_bytes":6721825,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":4842,"num_read_kb":14908,"num_write":316,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.14","version":"126'610","reported_seq":4188,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.794817+0000","last_change":"2024-09-21T15:35:07.795131+0000","last_active":"2024-09-21T15:35:07.794817+0000","last_peered":"2024-09-21T15:35:07.794817+0000","last_clean":"2024-09-21T15:35:07.794817+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:35:07.794817+0000","last_undegraded":"2024-09-21T15:35:07.794817+0000","last_fullsized":"2024-09-21T15:35:07.794817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0062231170000000002,"stat_sum":{"num_bytes":13568313,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":5984,"num_read_kb":15506,"num_write":753,"num_write_kb":21354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.760792+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:35:06.760792+0000","last_peered":"2024-09-21T15:35:06.760792+0000","last_clean":"2024-09-21T15:35:06.760792+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:35:06.760792+0000","last_undegraded":"2024-09-21T15:35:06.760792+0000","last_fullsized":"2024-09-21T15:35:06.760792+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"125'483","reported_seq":4496,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.774621+0000","last_change":"2024-09-21T15:35:08.768808+0000","last_active":"2024-09-21T15:35:07.774621+0000","last_peered":"2024-09-21T15:35:06.870360+0000","last_clean":"2024-09-21T15:35:06.870360+0000","last_became_active":"2024-09-21T15:32:46.733495+0000","last_became_peered":"2024-09-21T15:32:46.733495+0000","last_unstale":"2024-09-21T15:35:07.774621+0000","last_undegraded":"2024-09-21T15:35:07.774621+0000","last_fullsized":"2024-09-21T15:35:07.774621+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:55:04.444642+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013053613,"stat_sum":{"num_bytes":12039443,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":7243,"num_read_kb":13566,"num_write":430,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":123,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.752766+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:35:06.752766+0000","last_peered":"2024-09-21T15:35:06.752766+0000","last_clean":"2024-09-21T15:35:06.752766+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:35:06.752766+0000","last_undegraded":"2024-09-21T15:35:06.752766+0000","last_fullsized":"2024-09-21T15:35:06.752766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"125'274","reported_seq":1397,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.798175+0000","last_change":"2024-09-21T15:35:07.799123+0000","last_active":"2024-09-21T15:35:07.798175+0000","last_peered":"2024-09-21T15:35:07.798175+0000","last_clean":"2024-09-21T15:35:07.798175+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:35:07.798175+0000","last_undegraded":"2024-09-21T15:35:07.798175+0000","last_fullsized":"2024-09-21T15:35:07.798175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019867872000000002,"stat_sum":{"num_bytes":32374075,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":1729,"num_read_kb":23154,"num_write":337,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":124,"reported_epoch":126,"state":"peering","last_fresh":"2024-09-21T15:35:07.777581+0000","last_change":"2024-09-21T15:35:07.777948+0000","last_active":"2024-09-21T15:35:07.777581+0000","last_peered":"2024-09-21T15:35:06.770315+0000","last_clean":"2024-09-21T15:35:06.770315+0000","last_became_active":"2024-09-21T15:32:36.790016+0000","last_became_peered":"2024-09-21T15:32:36.790016+0000","last_unstale":"2024-09-21T15:35:07.777581+0000","last_undegraded":"2024-09-21T15:35:07.777581+0000","last_fullsized":"2024-09-21T15:35:07.777581+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:27.132848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"125'192","reported_seq":843,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.776650+0000","last_change":"2024-09-21T15:35:07.778985+0000","last_active":"2024-09-21T15:35:07.776650+0000","last_peered":"2024-09-21T15:35:07.776650+0000","last_clean":"2024-09-21T15:35:07.776650+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:35:07.776650+0000","last_undegraded":"2024-09-21T15:35:07.776650+0000","last_fullsized":"2024-09-21T15:35:07.776650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035087569999999999,"stat_sum":{"num_bytes":12859316,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":836,"num_read_kb":4075,"num_write":190,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":124,"reported_epoch":126,"state":"peering","last_fresh":"2024-09-21T15:35:07.774966+0000","last_change":"2024-09-21T15:35:07.775102+0000","last_active":"2024-09-21T15:35:07.774966+0000","last_peered":"2024-09-21T15:35:06.752509+0000","last_clean":"2024-09-21T15:35:06.752509+0000","last_became_active":"2024-09-21T15:32:36.785472+0000","last_became_peered":"2024-09-21T15:32:36.785472+0000","last_unstale":"2024-09-21T15:35:07.774966+0000","last_undegraded":"2024-09-21T15:35:07.774966+0000","last_fullsized":"2024-09-21T15:35:07.774966+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:47:31.078892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"125'708","reported_seq":1212,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.788507+0000","last_change":"2024-09-21T15:35:07.793133+0000","last_active":"2024-09-21T15:35:07.788507+0000","last_peered":"2024-09-21T15:35:07.788507+0000","last_clean":"2024-09-21T15:35:07.788507+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:35:07.788507+0000","last_undegraded":"2024-09-21T15:35:07.788507+0000","last_fullsized":"2024-09-21T15:35:07.788507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099211439999999998,"stat_sum":{"num_bytes":10089065,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":2981,"num_read_kb":10411,"num_write":1007,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.3","version":"125'306","reported_seq":6261,"reported_epoch":126,"state":"peering","last_fresh":"2024-09-21T15:35:07.776606+0000","last_change":"2024-09-21T15:35:07.777086+0000","last_active":"2024-09-21T15:35:07.776606+0000","last_peered":"2024-09-21T15:35:06.897399+0000","last_clean":"2024-09-21T15:35:06.897399+0000","last_became_active":"2024-09-21T15:32:46.723134+0000","last_became_peered":"2024-09-21T15:32:46.723134+0000","last_unstale":"2024-09-21T15:35:07.776606+0000","last_undegraded":"2024-09-21T15:35:07.776606+0000","last_fullsized":"2024-09-21T15:35:07.776606+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:00.553758+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0097377269999999998,"stat_sum":{"num_bytes":17294908,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":5911,"num_read_kb":16451,"num_write":316,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"2.2","version":"20'2","reported_seq":124,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.758743+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:35:06.758743+0000","last_peered":"2024-09-21T15:35:06.758743+0000","last_clean":"2024-09-21T15:35:06.758743+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:35:06.758743+0000","last_undegraded":"2024-09-21T15:35:06.758743+0000","last_fullsized":"2024-09-21T15:35:06.758743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"121'806","reported_seq":2468,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.779070+0000","last_change":"2024-09-21T15:35:07.779263+0000","last_active":"2024-09-21T15:35:07.779070+0000","last_peered":"2024-09-21T15:35:07.779070+0000","last_clean":"2024-09-21T15:35:07.779070+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:35:07.779070+0000","last_undegraded":"2024-09-21T15:35:07.779070+0000","last_fullsized":"2024-09-21T15:35:07.779070+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025466659999999999,"stat_sum":{"num_bytes":25050295,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":5673,"num_read_kb":8941,"num_write":1083,"num_write_kb":11847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.759690+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:35:06.759690+0000","last_peered":"2024-09-21T15:35:06.759690+0000","last_clean":"2024-09-21T15:35:06.759690+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:35:06.759690+0000","last_undegraded":"2024-09-21T15:35:06.759690+0000","last_fullsized":"2024-09-21T15:35:06.759690+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'488","reported_seq":2826,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.773215+0000","last_change":"2024-09-21T15:35:08.690231+0000","last_active":"2024-09-21T15:35:07.773215+0000","last_peered":"2024-09-21T15:35:05.755613+0000","last_clean":"2024-09-21T15:35:05.755613+0000","last_became_active":"2024-09-21T15:34:29.762828+0000","last_became_peered":"2024-09-21T15:34:29.762828+0000","last_unstale":"2024-09-21T15:35:07.773215+0000","last_undegraded":"2024-09-21T15:35:07.773215+0000","last_fullsized":"2024-09-21T15:35:07.773215+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:52.872437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023337430000000001,"stat_sum":{"num_bytes":18188363,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":4035,"num_read_kb":12474,"num_write":672,"num_write_kb":13933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2138700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.767974+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:35:06.767974+0000","last_peered":"2024-09-21T15:35:06.767974+0000","last_clean":"2024-09-21T15:35:06.767974+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:35:06.767974+0000","last_undegraded":"2024-09-21T15:35:06.767974+0000","last_fullsized":"2024-09-21T15:35:06.767974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"125'360","reported_seq":979,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.783494+0000","last_change":"2024-09-21T15:35:07.783494+0000","last_active":"2024-09-21T15:35:07.783494+0000","last_peered":"2024-09-21T15:35:07.783494+0000","last_clean":"2024-09-21T15:35:07.783494+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:35:07.783494+0000","last_undegraded":"2024-09-21T15:35:07.783494+0000","last_fullsized":"2024-09-21T15:35:07.783494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015669600000000001,"stat_sum":{"num_bytes":6833842,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":1576,"num_read_kb":4780,"num_write":393,"num_write_kb":6522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"2.3","version":"18'1","reported_seq":147,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-21T15:35:06.758565+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:35:06.758565+0000","last_peered":"2024-09-21T15:35:06.758565+0000","last_clean":"2024-09-21T15:35:06.758565+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:35:06.758565+0000","last_undegraded":"2024-09-21T15:35:06.758565+0000","last_fullsized":"2024-09-21T15:35:06.758565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":259,"reported_epoch":126,"state":"peering","last_fresh":"2024-09-21T15:35:07.774721+0000","last_change":"2024-09-21T15:35:07.775115+0000","last_active":"2024-09-21T15:35:07.774721+0000","last_peered":"2024-09-21T15:35:06.767758+0000","last_clean":"2024-09-21T15:35:06.767758+0000","last_became_active":"2024-09-21T15:34:33.799956+0000","last_became_peered":"2024-09-21T15:34:33.799956+0000","last_unstale":"2024-09-21T15:35:07.774721+0000","last_undegraded":"2024-09-21T15:35:07.774721+0000","last_fullsized":"2024-09-21T15:35:07.774721+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:36:47.014488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"120'554","reported_seq":1560,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.775272+0000","last_change":"2024-09-21T15:35:07.775722+0000","last_active":"2024-09-21T15:35:07.775272+0000","last_peered":"2024-09-21T15:35:07.775272+0000","last_clean":"2024-09-21T15:35:07.775272+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:35:07.775272+0000","last_undegraded":"2024-09-21T15:35:07.775272+0000","last_fullsized":"2024-09-21T15:35:07.775272+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066376800000000002,"stat_sum":{"num_bytes":11292668,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2932,"num_read_kb":7105,"num_write":783,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.9","version":"126'496","reported_seq":1672,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.792070+0000","last_change":"2024-09-21T15:35:07.790020+0000","last_active":"2024-09-21T15:35:07.792070+0000","last_peered":"2024-09-21T15:35:07.792070+0000","last_clean":"2024-09-21T15:35:07.792070+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:35:07.792070+0000","last_undegraded":"2024-09-21T15:35:07.792070+0000","last_fullsized":"2024-09-21T15:35:07.792070+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015597468,"stat_sum":{"num_bytes":24391047,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":3280,"num_read_kb":7983,"num_write":657,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.a","version":"126'305","reported_seq":4875,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.791309+0000","last_change":"2024-09-21T15:35:07.791973+0000","last_active":"2024-09-21T15:35:07.791309+0000","last_peered":"2024-09-21T15:35:07.791309+0000","last_clean":"2024-09-21T15:35:07.791309+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:35:07.791309+0000","last_undegraded":"2024-09-21T15:35:07.791309+0000","last_fullsized":"2024-09-21T15:35:07.791309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014850065000000001,"stat_sum":{"num_bytes":17696673,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4768,"num_read_kb":15793,"num_write":317,"num_write_kb":6694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.779156+0000","last_change":"2024-09-21T15:35:08.690381+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T15:35:07.779156+0000","last_undegraded":"2024-09-21T15:35:07.779156+0000","last_fullsized":"2024-09-21T15:35:07.779156+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:29:43.649077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"126'274","reported_seq":2096,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.794462+0000","last_change":"2024-09-21T15:35:07.794976+0000","last_active":"2024-09-21T15:35:07.794462+0000","last_peered":"2024-09-21T15:35:07.794462+0000","last_clean":"2024-09-21T15:35:07.794462+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:35:07.794462+0000","last_undegraded":"2024-09-21T15:35:07.794462+0000","last_fullsized":"2024-09-21T15:35:07.794462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012516935,"stat_sum":{"num_bytes":23866230,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":1844,"num_read_kb":12528,"num_write":257,"num_write_kb":23583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.d","version":"121'360","reported_seq":17409,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.796665+0000","last_change":"2024-09-21T15:35:07.798033+0000","last_active":"2024-09-21T15:35:07.796665+0000","last_peered":"2024-09-21T15:35:07.796665+0000","last_clean":"2024-09-21T15:35:07.796665+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:35:07.796665+0000","last_undegraded":"2024-09-21T15:35:07.796665+0000","last_fullsized":"2024-09-21T15:35:07.796665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021614815999999999,"stat_sum":{"num_bytes":14184493,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":17987,"num_read_kb":7703,"num_write":506,"num_write_kb":13560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.e","version":"126'330","reported_seq":37885,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.795156+0000","last_change":"2024-09-21T15:35:07.796071+0000","last_active":"2024-09-21T15:35:07.795156+0000","last_peered":"2024-09-21T15:35:07.795156+0000","last_clean":"2024-09-21T15:35:07.795156+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:35:07.795156+0000","last_undegraded":"2024-09-21T15:35:07.795156+0000","last_fullsized":"2024-09-21T15:35:07.795156+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021640415,"stat_sum":{"num_bytes":26061551,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":37992,"num_read_kb":47779,"num_write":350,"num_write_kb":22499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.f","version":"126'387","reported_seq":1752,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.801912+0000","last_change":"2024-09-21T15:35:07.802480+0000","last_active":"2024-09-21T15:35:07.801912+0000","last_peered":"2024-09-21T15:35:07.801912+0000","last_clean":"2024-09-21T15:35:07.801912+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:35:07.801912+0000","last_undegraded":"2024-09-21T15:35:07.801912+0000","last_fullsized":"2024-09-21T15:35:07.801912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.023616590999999999,"stat_sum":{"num_bytes":18848165,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":2570,"num_read_kb":8765,"num_write":453,"num_write_kb":5117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.10","version":"125'271","reported_seq":943,"reported_epoch":126,"state":"peering","last_fresh":"2024-09-21T15:35:07.775739+0000","last_change":"2024-09-21T15:35:07.776210+0000","last_active":"2024-09-21T15:35:07.775739+0000","last_peered":"2024-09-21T15:35:06.891029+0000","last_clean":"2024-09-21T15:35:06.891029+0000","last_became_active":"2024-09-21T15:34:58.962837+0000","last_became_peered":"2024-09-21T15:34:58.962837+0000","last_unstale":"2024-09-21T15:35:07.775739+0000","last_undegraded":"2024-09-21T15:35:07.775739+0000","last_fullsized":"2024-09-21T15:35:07.775739+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:55.319043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030987369999999998,"stat_sum":{"num_bytes":12898891,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":1160,"num_read_kb":7871,"num_write":250,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":4368836,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"3.11","version":"119'504","reported_seq":6324,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.781871+0000","last_change":"2024-09-21T15:35:07.781871+0000","last_active":"2024-09-21T15:35:07.781871+0000","last_peered":"2024-09-21T15:35:07.781871+0000","last_clean":"2024-09-21T15:35:07.781871+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:35:07.781871+0000","last_undegraded":"2024-09-21T15:35:07.781871+0000","last_fullsized":"2024-09-21T15:35:07.781871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082348860000000003,"stat_sum":{"num_bytes":25206458,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":8553,"num_read_kb":23034,"num_write":620,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]},{"pgid":"3.12","version":"125'551","reported_seq":7341,"reported_epoch":126,"state":"unknown","last_fresh":"2024-09-21T15:35:07.776184+0000","last_change":"2024-09-21T15:35:08.690674+0000","last_active":"2024-09-21T15:35:07.776184+0000","last_peered":"2024-09-21T15:35:06.768947+0000","last_clean":"2024-09-21T15:35:06.768947+0000","last_became_active":"2024-09-21T15:34:33.798598+0000","last_became_peered":"2024-09-21T15:34:33.798598+0000","last_unstale":"2024-09-21T15:35:07.776184+0000","last_undegraded":"2024-09-21T15:35:07.776184+0000","last_fullsized":"2024-09-21T15:35:07.776184+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:03.004684+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.018143943999999999,"stat_sum":{"num_bytes":10871771,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":7152,"num_read_kb":18559,"num_write":683,"num_write_kb":22833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":2138377,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"21","length":"4"}]},{"pgid":"3.13","version":"126'405","reported_seq":10431,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-21T15:35:07.796443+0000","last_change":"2024-09-21T15:35:07.796443+0000","last_active":"2024-09-21T15:35:07.796443+0000","last_peered":"2024-09-21T15:35:07.796443+0000","last_clean":"2024-09-21T15:35:07.796443+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:35:07.796443+0000","last_undegraded":"2024-09-21T15:35:07.796443+0000","last_fullsized":"2024-09-21T15:35:07.796443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.022180103999999999,"stat_sum":{"num_bytes":17795243,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":12671,"num_read_kb":33125,"num_write":364,"num_write_kb":15127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":581201236,"num_objects":1030,"num_object_clones":221,"num_object_copies":2060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":424,"num_read":225573,"num_read_kb":472713,"num_write":15399,"num_write_kb":357055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":884,"num_bytes_recovered":22756232,"num_keys_recovered":139,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":137216000,"data_stored":161711876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12589,"ondisk_log_size":12589,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55284,"kb_used_data":24160,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94316556,"statfs":{"total":96636764160,"available":96580153344,"internally_reserved":0,"allocated":24739840,"data_stored":27122492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.53500000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.622},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.05}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":1.006}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":0.95999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52736,"kb_used_data":13392,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94319104,"statfs":{"total":96636764160,"available":96582762496,"internally_reserved":0,"allocated":13713408,"data_stored":14200784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.56299999999999994}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.88}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.83799999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.63900000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":1.119}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":0.67300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65328,"kb_used_data":16072,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94306512,"statfs":{"total":96636764160,"available":96569868288,"internally_reserved":0,"allocated":16457728,"data_stored":18314723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.626},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":0.80500000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86299999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.73699999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.93100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61868,"kb_used_data":26656,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94309972,"statfs":{"total":96636764160,"available":96573411328,"internally_reserved":0,"allocated":27295744,"data_stored":29260207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.625},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":1.008}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.91600000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.872},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.84899999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.81499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77544,"kb_used_data":28288,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94294296,"statfs":{"total":96636764160,"available":96557359104,"internally_reserved":0,"allocated":28966912,"data_stored":34364138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.44},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53600000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.83499999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.85899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.71399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42284,"kb_used_data":7036,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94329556,"statfs":{"total":96636764160,"available":96593465344,"internally_reserved":0,"allocated":7204864,"data_stored":7205783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.91600000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.72799999999999998,"15min":0.75800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.613,"15min":1.613},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.874}]},{"osd":3,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.159,"15min":1.159},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.022},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90100000000000002,"15min":0.91800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1919999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.98199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.536,"15min":1.536},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1699999999999999,"5min":1.52,"15min":1.52},"last":0.95099999999999996}]},{"osd":6,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.94399999999999995,"15min":0.96999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0860000000000001,"5min":1.617,"15min":1.617},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.95199999999999996,"15min":0.97799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.995,"5min":1.554,"15min":1.554},"last":1.075}]},{"osd":7,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1200000000000001,"5min":1.669,"15min":1.669},"last":1.101},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2390000000000001,"5min":1.589,"15min":1.589},"last":1.0469999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":11288,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":11558912,"data_stored":12960288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.81999999999999995,"15min":0.79600000000000004},"min":{"1min":0.55000000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.55900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2390000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.86699999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2729999999999999,"5min":1.514,"15min":1.514},"last":0.748},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.67200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.891,"15min":1.891},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.94099999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3480000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.98899999999999999,"15min":1.004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3919999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.95699999999999996,"15min":0.93999999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":0.73299999999999998}]}]},{"osd":0,"up_from":0,"seq":55834574884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18432000,"data_stored":23851917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10047488,"data_stored":12249119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5697536,"data_stored":6494633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27459584,"data_stored":33652988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25788416,"data_stored":28549057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14946304,"data_stored":17603554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12206080,"data_stored":13489634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":22638592,"data_stored":25820974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:35:13.673 INFO:tasks.thrashosds.thrasher:Setting 3.b to [7, 6] 2024-09-21T15:35:13.673 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.b', '7', '6'] 2024-09-21T15:35:13.674 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.b 7 6 2024-09-21T15:35:13.675 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:13.671+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:13.703 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:13.704 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:13.704 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:13.704 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:13.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:13.776 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:13.774+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:13.876 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:13.874+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:13.897 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:13.897 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:13.897 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:13.897 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:13.897 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:13.912 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:13.977 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:13.974+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759752219000006, 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.039 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:14.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759728159999995, 2024-09-21T15:35:14.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:14.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:14.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759704197000005, 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.051 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759680431000007, 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:14.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759656789000005, 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.056 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:14.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759631236000004, 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.060 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:14.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759587021000002, 2024-09-21T15:35:14.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:14.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:14.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759558311999996, 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759526769000004, 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:14.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:14.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759499696999995, 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.077 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.759473764000006, 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:14.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.758938821000001, 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:14.085 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:14.089 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.758309152999999, 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:14.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:14.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.093 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754397589999996, 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:14.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754351091999993, 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:14.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754301923, 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:14.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:14.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754254075000006, 2024-09-21T15:35:14.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.110 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754205686000006, 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:14.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.114 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.754124765, 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.115 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:14.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:14.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:14.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 66.750526402000006, 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:14.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:14.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:14.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:14.123 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:14.123 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:14.125 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:14.125 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:14.119+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:14.225 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:14.222+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:14.266 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:14.266 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:14.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:14.295 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:14.309 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:14.313 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:14.314 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:14.330 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:14.331 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:14.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:14.347 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:14.357 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:14.355+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:14.365 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:14.365 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:14.381 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:14.426 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:14.423+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:14.432 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:14.432 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:14.447 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:14.526 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:14.522+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:14.613 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:14.613 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:14.627 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:14.627 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:14.626+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:14.728 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:14.726+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:14.749 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:14.749 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:14.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:14.829 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:14.826+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:14.833 INFO:teuthology.orchestra.run.smithi107.stderr:set 3.b pg_upmap mapping to [7,6] 2024-09-21T15:35:14.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:35:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:14.897 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:14.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:14.955 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:14.954+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:15.030 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:15.026+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.041 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.054 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:15.131 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:15.127+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.176 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.190 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:15.231 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:15.230+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:15.332 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:15.330+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:15.356 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.356 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.370 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:15.433 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:15.430+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:15.493 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.493 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:15.551 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:15.550+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:15.633 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.633 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.635 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:15.631+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:15.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:15.735 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:15.734+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:15.816 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.816 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:15.836 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:35:15.836 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:15.834+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:15.937 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:15.935+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:15.960 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:15.960 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:15.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:16.038 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:16.034+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:16.100 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:16.100 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:16.114 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:16.156 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:16.154+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:16.240 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:16.238+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:16.276 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:16.276 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:16.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:16.375 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:16.374+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:16.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:16.443 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:16.442+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:16.456 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:16.456 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:16.471 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:16.542 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:16.543 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:16.539+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:16.558 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:16.559 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:16.574 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:16.575 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:16.592 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:16.612 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:16.634 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:16.634 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:16.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:16.635 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:16.635 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:16.643 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:16.639+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:16.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:16.655 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:16.743 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:16.743+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:16.782 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:16.782 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:16.782 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:16.782 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:16.782 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:16.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:16.844 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:16.842+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:16.920 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 55.603653547, 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.921 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.922 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.924 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.925 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.685733467999999, 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.928 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.929 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.930 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.931 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.685699194000001, 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:16.932 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:16.933 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.934 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.935 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.685688934000002, 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.936 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.937 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.939 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681815839999999, 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:16.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:16.941 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.942 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.943 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681807072000002, 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.944 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.945 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.946 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.947 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681798386000001, 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.948 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.950 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:16.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681789939000002, 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.953 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.954 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.955 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681781397999998, 2024-09-21T15:35:16.956 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.957 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.958 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.959 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:16.960 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681772666000001, 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.961 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.964 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681763731, 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.965 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:16.966 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.681715092000001, 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.969 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.970 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.971 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.972 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677834331, 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:16.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.974 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.975 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.976 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677825492, 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.977 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:16.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:16.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677816630999999, 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:16.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.982 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.983 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.984 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.985 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677807911999999, 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.987 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.988 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677799427, 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.990 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.991 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.992 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677790690999998, 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:16.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.995 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.996 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.997 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:16.998 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677782167, 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:16.999 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.000 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 30.677773715000001, 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.006 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.007 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:17.008 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:17.008 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:17.007+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:17.109 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:17.106+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:17.177 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:17.177 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:17.177 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:17.177 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:17.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:17.235 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:17.234+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:17.311 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:17.307+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:17.367 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:17.367 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:17.367 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:17.367 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:17.367 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:17.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:17.411 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:17.407+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:17.513 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:17.511+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 145.14713060400001, 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:17.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.518 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.519 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.520 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:17.521 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 145.118919428, 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.523 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:17.524 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:17.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.92962005999999, 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:17.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.529 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.530 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:17.531 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.798235369, 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:17.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:17.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:17.535 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 140.63713796499999, 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:17.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:17.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:17.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:17.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 140.637085418, 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 140.63703729299999, 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.550 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 140.62464040500001, 2024-09-21T15:35:17.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:17.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.556 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 139.063867423, 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:17.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.99684305700001, 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:17.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:17.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:17.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:17.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:17.567 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.861722926, 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.569 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:17.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:17.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:17.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.82106078300001, 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.574 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.821047566, 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:17.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:17.581 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.82103475900001, 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:17.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:17.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:17.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:17.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.82099435200001, 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:17.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:17.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:17.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.591 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.82096988399999, 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:17.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:17.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.80174789599999, 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 138.714444313, 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:17.601 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.719758870000007, 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:17.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:17.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:17.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.305499522999995, 2024-09-21T15:35:17.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:17.612 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:17.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:17.614 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:17.615 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:17.616 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:17.614+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:17.716 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:17.711+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:17.745 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:17.745 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:17.745 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:17.745 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:17.759 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:17.818 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:17.814+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:17.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:35:17.918 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:17.915+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:17.924 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:17.924 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:17.924 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:17.924 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:17.924 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:17.940 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:18.019 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:18.014+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.113 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.834025873000002, 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.115 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.834001814000004, 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833977851, 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833954085000002, 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:18.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:18.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.129 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833930443, 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:18.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.132 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.133 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833904889999999, 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.137 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833860674999997, 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:18.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:18.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.141 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833831966000005, 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:18.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833800423, 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.146 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:18.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.149 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833773351000005, 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.150 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:18.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.153 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:18.154 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833747418000002, 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.156 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.157 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.158 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.833212474999996, 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.159 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:18.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:18.161 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.162 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.832582806999994, 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.163 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.165 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:18.166 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828671244000006, 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:18.167 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:18.168 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:18.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:18.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828624746000003, 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:18.171 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:18.172 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:18.173 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.174 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828575576999995, 2024-09-21T15:35:18.175 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.176 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.177 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.178 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:18.179 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828527729000001, 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.180 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.181 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828479340000001, 2024-09-21T15:35:18.183 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.184 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.185 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.186 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.828398418999996, 2024-09-21T15:35:18.187 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.188 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:18.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:18.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 70.824800056000001, 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.191 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:18.193 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:18.194 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.195 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:18.196 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:18.197 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:18.194+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:18.297 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:18.298+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:18.322 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:18.322 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:18.336 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:18.398 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:18.395+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:18.461 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:18.461 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:18.461 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:18.461 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:18.461 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:18.475 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:18.517 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:18.515+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:18.600 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:18.598+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360501752000005, 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:35:18.643 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:35:18.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:35:18.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360443191000002, 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.647 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:18.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:35:18.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:35:18.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360372475999995, 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.651 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:18.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360322840999999, 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:35:18.655 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.656 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.657 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.658 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:18.659 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360255073000005, 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.660 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.661 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.662 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:18.663 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360144199999993, 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.664 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.665 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:35:18.666 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.667 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360050232999996, 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.668 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.669 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:35:18.670 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:35:18.671 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.360002108000003, 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:35:18.672 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:35:18.673 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:35:18.674 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.675 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359932430000001, 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.676 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.677 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.678 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:35:18.679 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359883038000007, 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:35:18.680 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.681 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:35:18.682 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:35:18.683 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359836169000005, 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.684 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.685 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.686 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.687 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359789074000005, 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.688 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.689 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.690 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.691 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359740036000005, 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.692 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.693 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.694 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:35:18.695 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359672287999999, 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:35:18.696 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:35:18.697 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:35:18.698 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.699 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359623608999996, 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.700 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:35:18.701 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:35:18.702 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:35:18.703 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.359513325999998, 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.704 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:35:18.705 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:35:18.706 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:35:18.707 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.357264869000005, 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.708 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.709 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.710 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.711 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.357195118000007, 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.712 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.713 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.714 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.357105466999997, 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:18.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:35:18.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.718 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 71.356825685000004, 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:18.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.722 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.723 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:18.724 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:18.724 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:18.724 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:18.725 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:18.726 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:18.723+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:18.826 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:18.822+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:35:18.887 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:18.888 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:18.904 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:18.904 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:18.907 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:18.907 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:18.907 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:18.907 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:18.921 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:18.921 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:18.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:18.926 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:18.922+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:18.938 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:18.938 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:18.957 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:18.957 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:18.974 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:18.974 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:18.994 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:19.013 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:19.028 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:19.023+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:19.054 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.054 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:19.157 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:19.155+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:19.229 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:19.227+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:19.232 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.232 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.247 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:19.337 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:19.334+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:19.413 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.413 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:19.431 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:19.426+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:19.532 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:19.526+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:19.594 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.594 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:19.649 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:19.646+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:19.733 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:19.730+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:19.771 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.771 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.785 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:19.834 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:19.830+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:19.865 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:19.865 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:19.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:35:19.928 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:19.928 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:19.936 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:19.930+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:19.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:20.036 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:20.030+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:20.074 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.074 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.089 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:20.136 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:20.134+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:20.188 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:35:20.188 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:35:20.206 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-21T15:35:19.859425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515076530,"num_objects":874,"num_object_clones":190,"num_object_copies":1748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":417,"num_read":239767,"num_read_kb":499524,"num_write":16893,"num_write_kb":383907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1726,"num_bytes_recovered":45495166,"num_keys_recovered":230,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13984,"ondisk_log_size":13984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":376740,"kb_used_data":102084,"kb_used_omap":10,"kb_used_meta":274293,"kb_avail":660226140,"statfs":{"total":676457349120,"available":676071567360,"internally_reserved":0,"allocated":104534016,"data_stored":117325069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11149,"internal_metadata":280876147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88813201,"num_objects":-160,"num_object_clones":-34,"num_object_copies":-320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-160,"num_whiteouts":-23,"num_read":848,"num_read_kb":731,"num_write":230,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":2989020,"num_keys_recovered":11,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026807"},"pg_stats":[{"pgid":"3.1f","version":"136'647","reported_seq":5409,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.872049+0000","last_change":"2024-09-21T15:35:17.872049+0000","last_active":"2024-09-21T15:35:17.872049+0000","last_peered":"2024-09-21T15:35:17.872049+0000","last_clean":"2024-09-21T15:35:17.872049+0000","last_became_active":"2024-09-21T15:35:11.807889+0000","last_became_peered":"2024-09-21T15:35:11.807889+0000","last_unstale":"2024-09-21T15:35:17.872049+0000","last_undegraded":"2024-09-21T15:35:17.872049+0000","last_fullsized":"2024-09-21T15:35:17.872049+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:51:42.496725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001782276,"stat_sum":{"num_bytes":16304504,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":6977,"num_read_kb":8299,"num_write":979,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1e","version":"134'473","reported_seq":7248,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.851387+0000","last_change":"2024-09-21T15:35:17.851651+0000","last_active":"2024-09-21T15:35:17.851387+0000","last_peered":"2024-09-21T15:35:17.851387+0000","last_clean":"2024-09-21T15:35:17.851387+0000","last_became_active":"2024-09-21T15:33:03.923868+0000","last_became_peered":"2024-09-21T15:33:03.923868+0000","last_unstale":"2024-09-21T15:35:17.851387+0000","last_undegraded":"2024-09-21T15:35:17.851387+0000","last_fullsized":"2024-09-21T15:35:17.851387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:42:01.766240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000401548,"stat_sum":{"num_bytes":17516087,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":8768,"num_read_kb":21859,"num_write":583,"num_write_kb":26511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1d","version":"134'157","reported_seq":582,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.871697+0000","last_change":"2024-09-21T15:35:17.871697+0000","last_active":"2024-09-21T15:35:17.871697+0000","last_peered":"2024-09-21T15:35:17.871697+0000","last_clean":"2024-09-21T15:35:17.871697+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:35:17.871697+0000","last_undegraded":"2024-09-21T15:35:17.871697+0000","last_fullsized":"2024-09-21T15:35:17.871697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015779629999999999,"stat_sum":{"num_bytes":12750223,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":283,"num_read_kb":6246,"num_write":205,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1c","version":"136'294","reported_seq":5821,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.970936+0000","last_change":"2024-09-21T15:35:17.870866+0000","last_active":"2024-09-21T15:35:17.970936+0000","last_peered":"2024-09-21T15:35:17.970936+0000","last_clean":"2024-09-21T15:35:17.970936+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:35:17.970936+0000","last_undegraded":"2024-09-21T15:35:17.970936+0000","last_fullsized":"2024-09-21T15:35:17.970936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001174409,"stat_sum":{"num_bytes":41546130,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":5582,"num_read_kb":23006,"num_write":315,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1b","version":"135'378","reported_seq":9775,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.844318+0000","last_change":"2024-09-21T15:35:17.844589+0000","last_active":"2024-09-21T15:35:17.844318+0000","last_peered":"2024-09-21T15:35:17.844318+0000","last_clean":"2024-09-21T15:35:17.844318+0000","last_became_active":"2024-09-21T15:35:11.815839+0000","last_became_peered":"2024-09-21T15:35:11.815839+0000","last_unstale":"2024-09-21T15:35:17.844318+0000","last_undegraded":"2024-09-21T15:35:17.844318+0000","last_fullsized":"2024-09-21T15:35:17.844318+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:26.625542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072584600000000004,"stat_sum":{"num_bytes":8348012,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":10160,"num_read_kb":17277,"num_write":356,"num_write_kb":7281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.1a","version":"133'236","reported_seq":7041,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.951556+0000","last_change":"2024-09-21T15:35:17.844175+0000","last_active":"2024-09-21T15:35:17.951556+0000","last_peered":"2024-09-21T15:35:17.951556+0000","last_clean":"2024-09-21T15:35:17.951556+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:35:17.951556+0000","last_undegraded":"2024-09-21T15:35:17.951556+0000","last_fullsized":"2024-09-21T15:35:17.951556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068292199999999998,"stat_sum":{"num_bytes":7631140,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":6716,"num_read_kb":35921,"num_write":223,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.19","version":"136'607","reported_seq":33435,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.971886+0000","last_change":"2024-09-21T15:35:17.847147+0000","last_active":"2024-09-21T15:35:17.971886+0000","last_peered":"2024-09-21T15:35:17.971886+0000","last_clean":"2024-09-21T15:35:17.971886+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:35:17.971886+0000","last_undegraded":"2024-09-21T15:35:17.971886+0000","last_fullsized":"2024-09-21T15:35:17.971886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020876060000000001,"stat_sum":{"num_bytes":39533094,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":34649,"num_read_kb":22756,"num_write":831,"num_write_kb":6876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.18","version":"136'421","reported_seq":1071,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.846607+0000","last_change":"2024-09-21T15:35:17.846607+0000","last_active":"2024-09-21T15:35:17.846607+0000","last_peered":"2024-09-21T15:35:17.846607+0000","last_clean":"2024-09-21T15:35:17.846607+0000","last_became_active":"2024-09-21T15:35:11.803557+0000","last_became_peered":"2024-09-21T15:35:11.803557+0000","last_unstale":"2024-09-21T15:35:17.846607+0000","last_undegraded":"2024-09-21T15:35:17.846607+0000","last_fullsized":"2024-09-21T15:35:17.846607+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:34:34.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001600644,"stat_sum":{"num_bytes":6918971,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2057,"num_read_kb":4688,"num_write":574,"num_write_kb":13264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.17","version":"136'658","reported_seq":2243,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.970721+0000","last_change":"2024-09-21T15:35:17.872535+0000","last_active":"2024-09-21T15:35:17.970721+0000","last_peered":"2024-09-21T15:35:17.970721+0000","last_clean":"2024-09-21T15:35:17.970721+0000","last_became_active":"2024-09-21T15:35:11.808448+0000","last_became_peered":"2024-09-21T15:35:11.808448+0000","last_unstale":"2024-09-21T15:35:17.970721+0000","last_undegraded":"2024-09-21T15:35:17.970721+0000","last_fullsized":"2024-09-21T15:35:17.970721+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:31.699479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019745589999999999,"stat_sum":{"num_bytes":22029455,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":4134,"num_read_kb":11703,"num_write":958,"num_write_kb":7878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.16","version":"134'325","reported_seq":5912,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.870971+0000","last_change":"2024-09-21T15:35:17.871252+0000","last_active":"2024-09-21T15:35:17.870971+0000","last_peered":"2024-09-21T15:35:17.870971+0000","last_clean":"2024-09-21T15:35:17.870971+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:35:17.870971+0000","last_undegraded":"2024-09-21T15:35:17.870971+0000","last_fullsized":"2024-09-21T15:35:17.870971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001236441,"stat_sum":{"num_bytes":23178028,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":6971,"num_read_kb":11466,"num_write":322,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.15","version":"136'285","reported_seq":5151,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.846635+0000","last_change":"2024-09-21T15:35:17.846635+0000","last_active":"2024-09-21T15:35:17.846635+0000","last_peered":"2024-09-21T15:35:17.846635+0000","last_clean":"2024-09-21T15:35:17.846635+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:35:17.846635+0000","last_undegraded":"2024-09-21T15:35:17.846635+0000","last_fullsized":"2024-09-21T15:35:17.846635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049909800000000001,"stat_sum":{"num_bytes":4346145,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":4842,"num_read_kb":14908,"num_write":319,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.14","version":"136'616","reported_seq":4229,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.975416+0000","last_change":"2024-09-21T15:35:17.854000+0000","last_active":"2024-09-21T15:35:17.975416+0000","last_peered":"2024-09-21T15:35:17.975416+0000","last_clean":"2024-09-21T15:35:17.975416+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:35:17.975416+0000","last_undegraded":"2024-09-21T15:35:17.975416+0000","last_fullsized":"2024-09-21T15:35:17.975416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073269100000000005,"stat_sum":{"num_bytes":11996821,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":5984,"num_read_kb":15506,"num_write":757,"num_write_kb":21354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:16.889960+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:35:16.889960+0000","last_peered":"2024-09-21T15:35:16.889960+0000","last_clean":"2024-09-21T15:35:16.889960+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:35:16.889960+0000","last_undegraded":"2024-09-21T15:35:16.889960+0000","last_fullsized":"2024-09-21T15:35:16.889960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"135'492","reported_seq":4710,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.847379+0000","last_change":"2024-09-21T15:35:17.849230+0000","last_active":"2024-09-21T15:35:17.847379+0000","last_peered":"2024-09-21T15:35:17.847379+0000","last_clean":"2024-09-21T15:35:17.847379+0000","last_became_active":"2024-09-21T15:35:11.809783+0000","last_became_peered":"2024-09-21T15:35:11.809783+0000","last_unstale":"2024-09-21T15:35:17.847379+0000","last_undegraded":"2024-09-21T15:35:17.847379+0000","last_fullsized":"2024-09-21T15:35:17.847379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:28.514531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040783219999999997,"stat_sum":{"num_bytes":11472147,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":7243,"num_read_kb":13566,"num_write":431,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.319286+0000","last_change":"2024-09-21T15:32:38.790465+0000","last_active":"2024-09-21T15:35:17.319286+0000","last_peered":"2024-09-21T15:35:17.319286+0000","last_clean":"2024-09-21T15:35:17.319286+0000","last_became_active":"2024-09-21T15:32:36.786390+0000","last_became_peered":"2024-09-21T15:32:36.786390+0000","last_unstale":"2024-09-21T15:35:17.319286+0000","last_undegraded":"2024-09-21T15:35:17.319286+0000","last_fullsized":"2024-09-21T15:35:17.319286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:56:19.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069836799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"136'299","reported_seq":1529,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.848138+0000","last_change":"2024-09-21T15:35:17.848138+0000","last_active":"2024-09-21T15:35:17.848138+0000","last_peered":"2024-09-21T15:35:17.848138+0000","last_clean":"2024-09-21T15:35:17.848138+0000","last_became_active":"2024-09-21T15:33:03.919004+0000","last_became_peered":"2024-09-21T15:33:03.919004+0000","last_unstale":"2024-09-21T15:35:17.848138+0000","last_undegraded":"2024-09-21T15:35:17.848138+0000","last_fullsized":"2024-09-21T15:35:17.848138+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:39.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012560200000000001,"stat_sum":{"num_bytes":31325499,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":1897,"num_read_kb":23300,"num_write":356,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.295922+0000","last_change":"2024-09-21T15:35:08.793665+0000","last_active":"2024-09-21T15:35:17.295922+0000","last_peered":"2024-09-21T15:35:17.295922+0000","last_clean":"2024-09-21T15:35:17.295922+0000","last_became_active":"2024-09-21T15:35:08.792994+0000","last_became_peered":"2024-09-21T15:35:08.792994+0000","last_unstale":"2024-09-21T15:35:17.295922+0000","last_undegraded":"2024-09-21T15:35:17.295922+0000","last_fullsized":"2024-09-21T15:35:17.295922+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:25.765197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"136'200","reported_seq":886,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.847372+0000","last_change":"2024-09-21T15:35:17.847372+0000","last_active":"2024-09-21T15:35:17.847372+0000","last_peered":"2024-09-21T15:35:17.847372+0000","last_clean":"2024-09-21T15:35:17.847372+0000","last_became_active":"2024-09-21T15:33:21.068701+0000","last_became_peered":"2024-09-21T15:33:21.068701+0000","last_unstale":"2024-09-21T15:35:17.847372+0000","last_undegraded":"2024-09-21T15:35:17.847372+0000","last_fullsized":"2024-09-21T15:35:17.847372+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:08.735515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022671879999999998,"stat_sum":{"num_bytes":4956084,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":836,"num_read_kb":4075,"num_write":196,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.318878+0000","last_change":"2024-09-21T15:35:08.791000+0000","last_active":"2024-09-21T15:35:17.318878+0000","last_peered":"2024-09-21T15:35:17.318878+0000","last_clean":"2024-09-21T15:35:17.318878+0000","last_became_active":"2024-09-21T15:35:08.790639+0000","last_became_peered":"2024-09-21T15:35:08.790639+0000","last_unstale":"2024-09-21T15:35:17.318878+0000","last_undegraded":"2024-09-21T15:35:17.318878+0000","last_fullsized":"2024-09-21T15:35:17.318878+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:11:19.605793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"136'714","reported_seq":1255,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.871827+0000","last_change":"2024-09-21T15:35:17.871827+0000","last_active":"2024-09-21T15:35:17.871827+0000","last_peered":"2024-09-21T15:35:17.871827+0000","last_clean":"2024-09-21T15:35:17.871827+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:35:17.871827+0000","last_undegraded":"2024-09-21T15:35:17.871827+0000","last_fullsized":"2024-09-21T15:35:17.871827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017437609999999999,"stat_sum":{"num_bytes":6792850,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":2984,"num_read_kb":10412,"num_write":1013,"num_write_kb":5984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.3","version":"136'343","reported_seq":6560,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.946557+0000","last_change":"2024-09-21T15:35:17.845056+0000","last_active":"2024-09-21T15:35:17.946557+0000","last_peered":"2024-09-21T15:35:17.946557+0000","last_clean":"2024-09-21T15:35:17.946557+0000","last_became_active":"2024-09-21T15:35:11.810047+0000","last_became_peered":"2024-09-21T15:35:11.810047+0000","last_unstale":"2024-09-21T15:35:17.946557+0000","last_undegraded":"2024-09-21T15:35:17.946557+0000","last_fullsized":"2024-09-21T15:35:17.946557+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:13.202377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046061300000000002,"stat_sum":{"num_bytes":10356227,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":5952,"num_read_kb":16492,"num_write":351,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.2","version":"20'2","reported_seq":134,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:16.749084+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:35:16.749084+0000","last_peered":"2024-09-21T15:35:16.749084+0000","last_clean":"2024-09-21T15:35:16.749084+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:35:16.749084+0000","last_undegraded":"2024-09-21T15:35:16.749084+0000","last_fullsized":"2024-09-21T15:35:16.749084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"136'845","reported_seq":2614,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.976154+0000","last_change":"2024-09-21T15:35:17.850802+0000","last_active":"2024-09-21T15:35:17.976154+0000","last_peered":"2024-09-21T15:35:17.976154+0000","last_clean":"2024-09-21T15:35:17.976154+0000","last_became_active":"2024-09-21T15:33:24.080144+0000","last_became_peered":"2024-09-21T15:33:24.080144+0000","last_unstale":"2024-09-21T15:35:17.976154+0000","last_undegraded":"2024-09-21T15:35:17.976154+0000","last_fullsized":"2024-09-21T15:35:17.976154+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:59:17.915376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049715219999999999,"stat_sum":{"num_bytes":23272137,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":5876,"num_read_kb":9115,"num_write":1117,"num_write_kb":11858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":98,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:16.752200+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:35:16.752200+0000","last_peered":"2024-09-21T15:35:16.752200+0000","last_clean":"2024-09-21T15:35:16.752200+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:35:16.752200+0000","last_undegraded":"2024-09-21T15:35:16.752200+0000","last_fullsized":"2024-09-21T15:35:16.752200+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"134'507","reported_seq":3025,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.843831+0000","last_change":"2024-09-21T15:35:17.844657+0000","last_active":"2024-09-21T15:35:17.843831+0000","last_peered":"2024-09-21T15:35:17.843831+0000","last_clean":"2024-09-21T15:35:17.843831+0000","last_became_active":"2024-09-21T15:35:11.808267+0000","last_became_peered":"2024-09-21T15:35:11.808267+0000","last_unstale":"2024-09-21T15:35:17.843831+0000","last_undegraded":"2024-09-21T15:35:17.843831+0000","last_fullsized":"2024-09-21T15:35:17.843831+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:14.427530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060023,"stat_sum":{"num_bytes":16081955,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":4068,"num_read_kb":12499,"num_write":690,"num_write_kb":13937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.296551+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:35:17.296551+0000","last_peered":"2024-09-21T15:35:17.296551+0000","last_clean":"2024-09-21T15:35:17.296551+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:35:17.296551+0000","last_undegraded":"2024-09-21T15:35:17.296551+0000","last_fullsized":"2024-09-21T15:35:17.296551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"133'389","reported_seq":1114,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.870581+0000","last_change":"2024-09-21T15:35:17.870914+0000","last_active":"2024-09-21T15:35:17.870581+0000","last_peered":"2024-09-21T15:35:17.870581+0000","last_clean":"2024-09-21T15:35:17.870581+0000","last_became_active":"2024-09-21T15:33:03.923695+0000","last_became_peered":"2024-09-21T15:33:03.923695+0000","last_unstale":"2024-09-21T15:35:17.870581+0000","last_undegraded":"2024-09-21T15:35:17.870581+0000","last_fullsized":"2024-09-21T15:35:17.870581+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:09:24.891922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001307915,"stat_sum":{"num_bytes":5013003,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":1744,"num_read_kb":4926,"num_write":413,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"2.3","version":"18'1","reported_seq":157,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:16.749052+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:35:16.749052+0000","last_peered":"2024-09-21T15:35:16.749052+0000","last_clean":"2024-09-21T15:35:16.749052+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:35:16.749052+0000","last_undegraded":"2024-09-21T15:35:16.749052+0000","last_fullsized":"2024-09-21T15:35:16.749052+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":289,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.296040+0000","last_change":"2024-09-21T15:35:11.814721+0000","last_active":"2024-09-21T15:35:17.296040+0000","last_peered":"2024-09-21T15:35:17.296040+0000","last_clean":"2024-09-21T15:35:17.296040+0000","last_became_active":"2024-09-21T15:35:11.813712+0000","last_became_peered":"2024-09-21T15:35:11.813712+0000","last_unstale":"2024-09-21T15:35:17.296040+0000","last_undegraded":"2024-09-21T15:35:17.296040+0000","last_fullsized":"2024-09-21T15:35:17.296040+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:38:16.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"136'564","reported_seq":1606,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.848067+0000","last_change":"2024-09-21T15:35:17.848067+0000","last_active":"2024-09-21T15:35:17.848067+0000","last_peered":"2024-09-21T15:35:17.848067+0000","last_clean":"2024-09-21T15:35:17.848067+0000","last_became_active":"2024-09-21T15:34:29.764497+0000","last_became_peered":"2024-09-21T15:34:29.764497+0000","last_unstale":"2024-09-21T15:35:17.848067+0000","last_undegraded":"2024-09-21T15:35:17.848067+0000","last_fullsized":"2024-09-21T15:35:17.848067+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:02.506168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026096610000000001,"stat_sum":{"num_bytes":7390359,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":2934,"num_read_kb":7106,"num_write":789,"num_write_kb":5823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.9","version":"136'518","reported_seq":1777,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.968713+0000","last_change":"2024-09-21T15:35:17.844960+0000","last_active":"2024-09-21T15:35:17.968713+0000","last_peered":"2024-09-21T15:35:17.968713+0000","last_clean":"2024-09-21T15:35:17.968713+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:35:17.968713+0000","last_undegraded":"2024-09-21T15:35:17.968713+0000","last_fullsized":"2024-09-21T15:35:17.968713+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073119500000000002,"stat_sum":{"num_bytes":20425761,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":3375,"num_read_kb":8067,"num_write":669,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.a","version":"134'317","reported_seq":4921,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.851138+0000","last_change":"2024-09-21T15:35:17.851600+0000","last_active":"2024-09-21T15:35:17.851138+0000","last_peered":"2024-09-21T15:35:17.851138+0000","last_clean":"2024-09-21T15:35:17.851138+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:35:17.851138+0000","last_undegraded":"2024-09-21T15:35:17.851138+0000","last_fullsized":"2024-09-21T15:35:17.851138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056529400000000004,"stat_sum":{"num_bytes":11488780,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":4771,"num_read_kb":15796,"num_write":323,"num_write_kb":6694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.b","version":"135'377","reported_seq":9152,"reported_epoch":136,"state":"creating+peering","last_fresh":"2024-09-21T15:35:17.615275+0000","last_change":"2024-09-21T15:35:18.692679+0000","last_active":"2024-09-21T15:35:18.692679+0000","last_peered":"2024-09-21T15:35:17.615275+0000","last_clean":"2024-09-21T15:35:13.815751+0000","last_became_active":"2024-09-21T15:35:17.311542+0000","last_became_peered":"2024-09-21T15:35:17.311542+0000","last_unstale":"2024-09-21T15:35:17.615275+0000","last_undegraded":"2024-09-21T15:35:17.615275+0000","last_fullsized":"2024-09-21T15:35:16.754826+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061121099999999996,"stat_sum":{"num_bytes":27128686,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":10467,"num_read_kb":20688,"num_write":385,"num_write_kb":11082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"1"}]},{"pgid":"3.c","version":"126'274","reported_seq":2131,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.869774+0000","last_change":"2024-09-21T15:35:17.870337+0000","last_active":"2024-09-21T15:35:17.869774+0000","last_peered":"2024-09-21T15:35:17.869774+0000","last_clean":"2024-09-21T15:35:17.869774+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:35:17.869774+0000","last_undegraded":"2024-09-21T15:35:17.869774+0000","last_fullsized":"2024-09-21T15:35:17.869774+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068472800000000001,"stat_sum":{"num_bytes":23866230,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":1844,"num_read_kb":12528,"num_write":257,"num_write_kb":23583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.d","version":"136'376","reported_seq":17464,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.967094+0000","last_change":"2024-09-21T15:35:17.846949+0000","last_active":"2024-09-21T15:35:17.967094+0000","last_peered":"2024-09-21T15:35:17.967094+0000","last_clean":"2024-09-21T15:35:17.967094+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:35:17.967094+0000","last_undegraded":"2024-09-21T15:35:17.967094+0000","last_fullsized":"2024-09-21T15:35:17.967094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073543999999999997,"stat_sum":{"num_bytes":11706256,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":18018,"num_read_kb":7727,"num_write":524,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.e","version":"134'359","reported_seq":38021,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.847163+0000","last_change":"2024-09-21T15:35:17.847517+0000","last_active":"2024-09-21T15:35:17.847163+0000","last_peered":"2024-09-21T15:35:17.847163+0000","last_clean":"2024-09-21T15:35:17.847163+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:35:17.847163+0000","last_undegraded":"2024-09-21T15:35:17.847163+0000","last_fullsized":"2024-09-21T15:35:17.847163+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001029147,"stat_sum":{"num_bytes":22306771,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":38168,"num_read_kb":47931,"num_write":370,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.f","version":"136'402","reported_seq":1808,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.848308+0000","last_change":"2024-09-21T15:35:17.848308+0000","last_active":"2024-09-21T15:35:17.848308+0000","last_peered":"2024-09-21T15:35:17.848308+0000","last_clean":"2024-09-21T15:35:17.848308+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:35:17.848308+0000","last_undegraded":"2024-09-21T15:35:17.848308+0000","last_fullsized":"2024-09-21T15:35:17.848308+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00103921,"stat_sum":{"num_bytes":18385947,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":2604,"num_read_kb":8790,"num_write":470,"num_write_kb":5122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.10","version":"136'283","reported_seq":1173,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.932518+0000","last_change":"2024-09-21T15:35:17.852468+0000","last_active":"2024-09-21T15:35:17.932518+0000","last_peered":"2024-09-21T15:35:17.932518+0000","last_clean":"2024-09-21T15:35:17.932518+0000","last_became_active":"2024-09-21T15:35:11.820200+0000","last_became_peered":"2024-09-21T15:35:11.820200+0000","last_unstale":"2024-09-21T15:35:17.932518+0000","last_undegraded":"2024-09-21T15:35:17.932518+0000","last_fullsized":"2024-09-21T15:35:17.932518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:08:24.236189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000648184,"stat_sum":{"num_bytes":8373746,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":1163,"num_read_kb":7872,"num_write":256,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.11","version":"136'516","reported_seq":6370,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.968823+0000","last_change":"2024-09-21T15:35:17.847623+0000","last_active":"2024-09-21T15:35:17.968823+0000","last_peered":"2024-09-21T15:35:17.968823+0000","last_clean":"2024-09-21T15:35:17.968823+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:35:17.968823+0000","last_undegraded":"2024-09-21T15:35:17.968823+0000","last_fullsized":"2024-09-21T15:35:17.968823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389068,"stat_sum":{"num_bytes":20160186,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":8553,"num_read_kb":23034,"num_write":626,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.12","version":"136'585","reported_seq":7688,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.973949+0000","last_change":"2024-09-21T15:35:17.848012+0000","last_active":"2024-09-21T15:35:17.973949+0000","last_peered":"2024-09-21T15:35:17.973949+0000","last_clean":"2024-09-21T15:35:17.973949+0000","last_became_active":"2024-09-21T15:35:11.810093+0000","last_became_peered":"2024-09-21T15:35:11.810093+0000","last_unstale":"2024-09-21T15:35:17.973949+0000","last_undegraded":"2024-09-21T15:35:17.973949+0000","last_fullsized":"2024-09-21T15:35:17.973949+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:43:40.024602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001314581,"stat_sum":{"num_bytes":5293202,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":7360,"num_read_kb":18740,"num_write":708,"num_write_kb":22839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]},{"pgid":"3.13","version":"134'407","reported_seq":10468,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-21T15:35:17.850011+0000","last_change":"2024-09-21T15:35:17.850555+0000","last_active":"2024-09-21T15:35:17.850011+0000","last_peered":"2024-09-21T15:35:17.850011+0000","last_clean":"2024-09-21T15:35:17.850011+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:35:17.850011+0000","last_undegraded":"2024-09-21T15:35:17.850011+0000","last_fullsized":"2024-09-21T15:35:17.850011+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075483099999999997,"stat_sum":{"num_bytes":16591707,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":12671,"num_read_kb":33125,"num_write":366,"num_write_kb":15127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":514486143,"num_objects":870,"num_object_clones":190,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":417,"num_read":239651,"num_read_kb":499424,"num_write":16732,"num_write_kb":381489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106536960,"data_stored":130793891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13864,"ondisk_log_size":13864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58396,"kb_used_data":23188,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94313444,"statfs":{"total":96636764160,"available":96576966656,"internally_reserved":0,"allocated":23744512,"data_stored":26362340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.73399999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":9.5950000000000006,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.86799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1180000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.02,"5min":1.252,"15min":1.252},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.85399999999999998,"15min":0.874},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0229999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":0.88400000000000001,"15min":0.81799999999999995},"min":{"1min":0.48499999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1679999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.629}]},{"osd":4,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.84599999999999997,"15min":0.874},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.087,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.347,"5min":0.94299999999999995,"15min":0.876},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":9.7490000000000006,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1120000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.90300000000000002,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.276,"5min":1.365,"15min":1.365},"last":0.47999999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.97999999999999998,"15min":1.0109999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1850000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.085,"15min":1.0349999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6799999999999997,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":0.53400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50948,"kb_used_data":11632,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94320892,"statfs":{"total":96636764160,"available":96584593408,"internally_reserved":0,"allocated":11911168,"data_stored":12837744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.501},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99199999999999999,"5min":1.23,"15min":1.23},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85099999999999998,"15min":0.875},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":1.452,"15min":1.452},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2130000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0329999999999999,"5min":1.425,"15min":1.425},"last":0.63400000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86699999999999999,"15min":0.877},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.02,"5min":1.228,"15min":1.228},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.441,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.117,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.264,"15min":1.264},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.91000000000000003,"15min":0.91900000000000004},"min":{"1min":0.39900000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2789999999999999,"5min":1.45,"15min":1.45},"last":0.61499999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.48799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.1759999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.48399999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2869999999999999,"5min":1.7,"15min":1.7},"last":0.73399999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.0449999999999999,"15min":1.0209999999999999},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.46700000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.256,"5min":1.843,"15min":1.843},"last":0.76300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63476,"kb_used_data":10216,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94308364,"statfs":{"total":96636764160,"available":96571764736,"internally_reserved":0,"allocated":10461184,"data_stored":11983715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.495,"15min":0.495},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.73999999999999999}]},{"osd":2,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.272,"15min":1.272},"last":0.57699999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.236,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2589999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.89600000000000002,"15min":0.90600000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.72599999999999998}]},{"osd":6,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.89400000000000002,"15min":0.90400000000000003},"min":{"1min":0.38,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.365,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.393,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97299999999999998,"15min":0.98099999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.875},{"interface":"front","average":{"1min":1.044,"5min":1.0249999999999999,"15min":1.022},"min":{"1min":0.42699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.89000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52416,"kb_used_data":17200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94319424,"statfs":{"total":96636764160,"available":96583090176,"internally_reserved":0,"allocated":17612800,"data_stored":19198410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.5089999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.069,"5min":3.069,"15min":3.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.92200000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.105}]},{"osd":3,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.82299999999999995,"15min":0.84699999999999998},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97599999999999998,"5min":2.048,"15min":2.048},"last":1.258},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.88700000000000001,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.0349999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.88100000000000001,"15min":0.89400000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":3.0289999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.754},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.89600000000000002,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":6.3369999999999997,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.004,"15min":1.008},"min":{"1min":0.42699999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.80300000000000005}]},{"osd":7,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.871,"15min":0.873},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.024,"5min":1.028,"15min":1.0289999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.2839999999999998,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76656,"kb_used_data":27284,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94295184,"statfs":{"total":96636764160,"available":96558268416,"internally_reserved":0,"allocated":27938816,"data_stored":33914303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.59499999999999997,"15min":0.623},"min":{"1min":0.27100000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.91200000000000003,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.51300000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.67800000000000005,"15min":0.69999999999999996},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2290000000000001,"5min":1.802,"15min":1.802},"last":0.315},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1990000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.42399999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.036,"5min":1.587,"15min":1.587},"last":0.59299999999999997},{"interface":"front","average":{"1min":2.1299999999999999,"5min":1.1899999999999999,"15min":1.034},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":24.064,"5min":24.064,"15min":24.064},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.89700000000000002,"15min":0.92900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.161,"5min":1.966,"15min":1.966},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.86599999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.879,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88400000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.67700000000000005}]},{"osd":7,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.97099999999999997,"15min":0.98699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3540000000000001,"5min":1.772,"15min":1.772},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.98999999999999999,"15min":1.024},"min":{"1min":0.51400000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.0589999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.69299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39716,"kb_used_data":4496,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332124,"statfs":{"total":96636764160,"available":96596094976,"internally_reserved":0,"allocated":4603904,"data_stored":4065632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48299999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.72799999999999998,"15min":0.75800000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.613,"15min":1.613},"last":0.441},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.57599999999999996,"15min":0.58799999999999997},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81999999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.159,"15min":1.159},"last":0.39300000000000002}]},{"osd":4,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.88700000000000001,"15min":0.91400000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.95299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90100000000000002,"15min":0.91800000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1919999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.165,"5min":1.536,"15min":1.536},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92800000000000005,"15min":0.95599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1699999999999999,"5min":1.52,"15min":1.52},"last":0.75}]},{"osd":6,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.94399999999999995,"15min":0.96999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0860000000000001,"5min":1.617,"15min":1.617},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.95199999999999996,"15min":0.97799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.995,"5min":1.554,"15min":1.554},"last":0.61899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.97599999999999998,"15min":0.998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1200000000000001,"5min":1.669,"15min":1.669},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.97199999999999998,"15min":0.996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.2390000000000001,"5min":1.589,"15min":1.589},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35132,"kb_used_data":8068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336708,"statfs":{"total":96636764160,"available":96600788992,"internally_reserved":0,"allocated":8261632,"data_stored":8962925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.35799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.81999999999999995,"15min":0.79600000000000004},"min":{"1min":0.55000000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.55900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2390000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.86499999999999999,"15min":0.873},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2729999999999999,"5min":1.514,"15min":1.514},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.89500000000000002,"15min":0.871},"min":{"1min":0.67200000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.73499999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3080000000000001,"5min":1.891,"15min":1.891},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.94099999999999995,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.3480000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.82699999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.98899999999999999,"15min":1.004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3919999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":1.018},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.95699999999999996,"15min":0.93999999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":3.8359999999999999,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.2090000000000001}]}]},{"osd":0,"up_from":0,"seq":55834574886,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18432000,"data_stored":23851917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6627328,"data_stored":8189195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":3291921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25210880,"data_stored":32063951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15982592,"data_stored":18424699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8826880,"data_stored":11209985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9601024,"data_stored":11397031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18882560,"data_stored":22365192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:35:20.209 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.210 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.223 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:20.272 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:20.270+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:20.338 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:20.334+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.407 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:20.440 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:20.434+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:20.540 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:20.538+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.574 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:20.641 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:20.638+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:20.710 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.710 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.724 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:20.767 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:20.766+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:20.843 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:20.842+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:20.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:35:20.897 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:20.897 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:20.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:20.944 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:20.938+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:21.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:21.045 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:21.042+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:21.077 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:21.077 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:21.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:21.145 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:21.142+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:21.172 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:21.189 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:21.189 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:21.204 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:21.205 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 59.896362234999998, 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:21.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.218 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.978442156, 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.978407881999999, 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:21.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:21.227 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.978397622000003, 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:21.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:21.231 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974524528000003, 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:21.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974515760000003, 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.236 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:21.240 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974507074000002, 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.244 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974498627000003, 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.245 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974490086000003, 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:21.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:21.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974481353999998, 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:21.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.257 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974472419000001, 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.974423780000002, 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:21.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970543018999997, 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:21.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.269 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:21.270 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970534180000001, 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:21.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970525319000004, 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970516600000003, 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:21.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:21.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970508115000001, 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.283 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:21.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970499379000003, 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:21.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.288 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.291 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970490855000001, 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.295 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 34.970482402999998, 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.296 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:21.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:21.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.300 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:21.301 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:21.301 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:21.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:21.302 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:21.302 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:21.303 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:21.303 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:21.303 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:21.303 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:21.304 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:21.298+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:21.305 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:21.306 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:21.403 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:21.402+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:21.422 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:21.422 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:21.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:21.503 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:21.502+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:21.557 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:21.557 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:21.557 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:21.571 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:21.616 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:21.614+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:21.705 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:21.702+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 149.37313989699999, 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:21.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:21.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:21.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.748 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 149.344928721, 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:21.749 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:21.750 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:21.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:21.752 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 148.15562935299999, 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.753 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.754 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.755 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 148.024244662, 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.758 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.759 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.760 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:21.761 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 144.863147258, 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.763 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:21.764 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:21.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:21.766 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 144.863094711, 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:21.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.768 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.769 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:21.770 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:21.771 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 144.863046586, 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.773 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.774 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:21.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.777 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 144.85064969800001, 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.778 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:21.779 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:21.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:21.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:21.782 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.783 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.28987671600001, 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.784 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.785 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.786 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.787 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.788 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.22285235000001, 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.789 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:21.791 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:21.792 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:21.793 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.794 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.087732219, 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.795 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:21.797 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.798 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.799 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.800 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.04707007600001, 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.801 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.802 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.803 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.804 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.04705685900001, 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.805 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:21.806 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.807 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.04704405199999, 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.809 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:21.811 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:21.812 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.04700364499999, 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.815 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.817 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.046979177, 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:21.818 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.819 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.820 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.821 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 143.027757189, 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:21.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:21.823 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:21.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:21.825 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.826 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:21.827 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 142.94045360600001, 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.828 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.829 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.830 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:21.831 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.945768162999997, 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.834 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:21.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 74.531508815999999, 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:21.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.838 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.839 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:21.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:21.842 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:35:21.843 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:21.844 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:21.838+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:21.944 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:21.942+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:21.986 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:21.986 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:21.986 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.001 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:22.044 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:22.042+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:22.121 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:22.122 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:22.122 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:22.122 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:22.122 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:22.181 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:22.178+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:22.246 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:22.242+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980957889999999, 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.259 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:22.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:22.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980933831000002, 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:22.263 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.267 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980909867999998, 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.269 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:22.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980886101999999, 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.272 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:22.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:22.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980862459999997, 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:22.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.278 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:22.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980836906999997, 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.281 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.282 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:22.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:22.284 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980792691999994, 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:22.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:22.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:22.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:22.288 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980763983000003, 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.289 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:22.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:22.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980732439999997, 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.293 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:22.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980705368000002, 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:22.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:22.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:22.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.300 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980679434999999, 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.301 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.304 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.980144491999994, 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.979514824000006, 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:22.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:22.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975603261000003, 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:22.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.316 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975556763, 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:22.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.320 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975507594000007, 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975459745999999, 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.325 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.326 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.327 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.328 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975411356999999, 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.329 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.330 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.331 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.332 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.975330435999993, 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.333 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.334 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.335 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.336 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.971732072999998, 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:22.337 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:22.338 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:22.339 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:22.340 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.341 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.342 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:22.347 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:22.346+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:22.449 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:22.446+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:22.510 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:22.510 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:22.510 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:22.510 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:22.550 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:22.546+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:22.645 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:22.645 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:22.645 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:22.645 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:22.645 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.651 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:22.650+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:22.659 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:22.752 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:22.750+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495786574999997, 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:35:22.778 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.779 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:35:22.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:35:22.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495728013999994, 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:35:22.782 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:35:22.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.786 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495657299000001, 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:35:22.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:35:22.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.790 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495607664000005, 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.791 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:35:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.795 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495539895999997, 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.796 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:35:22.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:35:22.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495429023, 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.800 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.801 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:35:22.802 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.803 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.804 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495335056000002, 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:35:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.807 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.808 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495286930999995, 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.809 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.810 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.811 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:22.812 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495217253000007, 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.813 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.814 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:35:22.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.816 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495167860999999, 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:22.817 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:35:22.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:35:22.819 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.820 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495120991999997, 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.821 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:35:22.822 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:35:22.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.824 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495073896999997, 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.825 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.827 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.828 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.495024858999997, 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.829 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:35:22.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:35:22.832 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.494957111000005, 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.833 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.834 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.835 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.836 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.494908432000003, 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.837 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:35:22.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:35:22.840 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.494798149000005, 2024-09-21T15:35:22.841 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:35:22.843 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.844 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.845 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.492549691999997, 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.846 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.847 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.848 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:35:22.849 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.492479940999999, 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.850 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:35:22.851 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:35:22.852 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:35:22.853 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.492390290000003, 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.854 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:35:22.855 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.856 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:35:22.857 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 75.492110507999996, 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:35:22.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:35:22.859 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.860 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.861 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.862 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.862 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:22.862 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:22.862 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:22.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:22.863 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:22.862+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:22.964 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:22.962+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:23.035 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:23.036 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:23.036 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:23.036 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:23.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:23.093 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:23.090+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:23.166 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:23.162+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:23.170 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:23.170 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:23.170 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:23.170 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:23.170 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:23.184 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:23.277 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:23.274+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:23.325 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.354 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.695035297999993, 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.355 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:35:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:35:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.691184715999995, 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.359 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.360 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.361 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.362 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.691159524, 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.363 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.365 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:35:23.366 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.687183501999996, 2024-09-21T15:35:23.367 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:35:23.369 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.370 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.687157479999996, 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.372 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.373 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:35:23.374 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.375 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.687133927000005, 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.376 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.377 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.378 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:35:23.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.687099085, 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.380 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:35:23.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.683117562999996, 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.384 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:35:23.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:35:23.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.683010186000004, 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:35:23.388 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.392 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.682983617000005, 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:35:23.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:35:23.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.682956040999997, 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:35:23.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.682932531000006, 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.401 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:35:23.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:35:23.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.404 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679230904999997, 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:35:23.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679201152000005, 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679174083000007, 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:35:23.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:35:23.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679150746999994, 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.417 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.421 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679082143000002, 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:23.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:35:23.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:35:23.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.425 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.679053402999998, 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:35:23.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.429 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.675140420000005, 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:35:23.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.433 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 74.675085347999996, 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:23.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:35:23.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:23.437 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:23.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:23.439 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:23.434+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:23.479 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0: {} 2024-09-21T15:35:23.479 INFO:teuthology.orchestra.run.smithi107.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T15:35:23.495 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:23.495 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:23.511 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:23.511 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:23.529 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:23.529 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:23.539 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:23.534+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:23.547 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:23.547 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:23.564 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:23.582 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:23.582 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:23.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:23.602 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:23.609 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:23.609 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:23.623 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:23.641 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:23.638+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:23.742 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:23.738+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:23.791 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:23.791 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:23.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:23.842 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:23.838+0000 7f22df7fe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26304) UID: 0 2024-09-21T15:35:23.845 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:35:23.934 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:23.935 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:23.943 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:23.938+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:23.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:24.044 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:24.042+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:24.079 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.079 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:24.145 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:24.142+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:24.246 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:24.242+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:24.248 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.249 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:24.347 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:24.342+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:24.425 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.425 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.439 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:24.448 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:24.446+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:24.549 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:24.546+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:24.563 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.563 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.577 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:24.649 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:24.646+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.695 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.709 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:24.750 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:24.746+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:24.851 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:24.846+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:24.874 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:24.874 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:24.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:24.913 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:35:24.952 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:24.946+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:25.034 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.034 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:25.052 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:25.050+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:25.153 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:25.150+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:25.178 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.178 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:25.209 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:25.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:25.210 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:25.210 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T15:35:25.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:35:25.254 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:25.250+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:25.306 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T15:35:25.306 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-21T15:35:25.306 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:35:25.306 DEBUG:teuthology.orchestra.run.smithi107:> type ceph-objectstore-tool 2024-09-21T15:35:25.310 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T15:35:25.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-21T15:35:25.321 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.321 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.336 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:25.381 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:25.378+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:25.455 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:25.450+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:25.469 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.469 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.484 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:25.556 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:25.554+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:25.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:25.643 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.643 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:25.663 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:25.662+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:25.758 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:25.754+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:25.778 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.778 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.792 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:25.859 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:25.854+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:25.959 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:25.958+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:25.965 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:25.965 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:25.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:26.061 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:26.058+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:26.109 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.109 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.123 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:26.161 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:26.158+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.243 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:26.262 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:26.258+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:26.362 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:26.358+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:26.373 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.374 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.387 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:26.464 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:26.458+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:26.503 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.503 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:26.609 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:26.606+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:26.665 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:26.662+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:26.685 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.685 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.700 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:26.793 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:26.790+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:26.868 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:26.866+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:26.875 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:26.875 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:26.891 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:26.936 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:35:26.981 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:26.978+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:27.063 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.064 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.069 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:27.066+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:27.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:27.168 INFO:tasks.thrashosds.thrasher:No PGs found for osd.0 2024-09-21T15:35:27.169 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:27.166+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:27.241 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.241 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.256 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:27.297 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:27.294+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:27.372 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:27.370+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:27.374 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.374 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:27.473 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:27.470+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:27.509 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.509 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.523 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:27.574 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:27.570+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:27.645 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.645 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:27.704 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:27.702+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:27.775 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:27.774+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:27.826 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.826 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:27.876 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:27.870+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:27.932 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:35:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:27.977 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:27.974+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:27.984 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:28.077 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:28.074+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:28.156 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.157 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:28.179 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:28.174+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:28.279 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:28.274+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:28.340 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.340 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:28.399 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:28.398+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:28.473 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.473 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.481 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:28.478+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:28.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:28.582 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:28.578+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:28.656 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.656 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.670 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:28.711 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:28.710+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:28.783 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:28.782+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:28.801 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.801 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.816 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:28.884 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:28.878+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:28.939 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:28.939 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:28.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:28.985 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:28.982+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:29.086 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:29.082+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:29.132 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.133 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.147 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:29.193 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:29.190+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:29.288 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:29.282+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:29.314 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.314 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.328 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:29.389 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:29.386+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:29.490 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:29.490+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:29.497 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.497 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:29.591 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:29.590+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:29.632 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.633 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.647 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:29.693 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:29.690+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:29.769 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.770 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:29.829 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:29.826+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:29.894 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:29.890+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:29.900 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:29.900 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:29.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:29.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:35:30.001 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:29.998+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:30.083 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:30.083 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:30.097 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:30.094+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:30.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:30.197 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:30.194+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:30.262 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:30.262 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:30.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:30.321 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:30.318+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:30.399 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:30.394+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:30.443 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:30.443 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:30.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:30.500 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:30.498+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:30.601 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:30.602+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:30.626 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:30.626 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:30.640 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:30.702 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:30.698+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:30.756 INFO:teuthology.orchestra.run.smithi107.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T15:35:30.782 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:30.783 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:30.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:30.787 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:30.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:30.805 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:30.806 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:30.806 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:30.802+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:30.822 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:30.822 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:30.842 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:30.842 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:30.863 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:30.884 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:30.904 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:30.902+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:30.904 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:30.908 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:35:30.908 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:35:30.925 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:30.925 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:30.925 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:30.925 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:30.925 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:30.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:30.948 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:35:31.019 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:31.018+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 135.948502725, 2024-09-21T15:35:31.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:31.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.50634651199999, 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:35:31.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.506321766, 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:35:31.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.506276816, 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:35:31.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:35:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:35:31.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.094 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.50242015800001, 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.095 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.502375798, 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.502322449, 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.103 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:31.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:35:31.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.502291056, 2024-09-21T15:35:31.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.111 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.50224532999999, 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:31.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:35:31.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.502215957, 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:35:31.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:35:31.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.502191397, 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.50216476, 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:31.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:35:31.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498326193, 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:35:31.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:31.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:35:31.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:35:31.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498296373, 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498270843, 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:35:31.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:31.141 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498245137, 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.143 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:35:31.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.49819509700001, 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:35:31.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.147 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:31.149 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498160297, 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.151 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.498135648, 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:35:31.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 111.49408524099999, 2024-09-21T15:35:31.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.161 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:31.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:31.163 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:31.158+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:31.264 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:31.262+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:31.332 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:31.332 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:31.332 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:31.332 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:31.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:31.365 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:31.362+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:31.466 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:31.466+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:31.517 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:31.517 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:31.517 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:31.517 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:31.517 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:31.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:31.583 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:31.582+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 70.336852941000004, 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:31.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.418932861999998, 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.661 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:31.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.418898587999998, 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.665 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:31.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:31.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.418888328000001, 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:31.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:31.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.415015234000002, 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:31.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:31.677 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.415006466000001, 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:31.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:31.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.41499778, 2024-09-21T15:35:31.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:31.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:31.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.414989333000001, 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:31.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.690 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.414980792000001, 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:31.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:31.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.694 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.414972059999997, 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:31.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:31.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.414963125, 2024-09-21T15:35:31.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:31.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.703 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.414914486000001, 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:31.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:31.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.411033725000003, 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.708 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:31.711 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.411024886, 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:31.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:31.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:31.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.411016025000002, 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.716 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.411007306000002, 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.410998821, 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.410990085000002, 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:31.728 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:31.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:31.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.731 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.410981561, 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:31.732 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:31.733 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.735 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 45.410973108999997, 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.736 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.739 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:31.740 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:31.740 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:31.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:31.741 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:31.738+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:31.842 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:31.838+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:31.922 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:31.922 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:31.922 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:31.922 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:31.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:31.943 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:31.938+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:32.044 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:32.038+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:32.060 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:32.060 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:32.060 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:32.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:32.145 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:32.142+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:32.169 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:32.169 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:32.169 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T15:35:32.169 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-21T15:35:32.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2024-09-21T15:35:32.200 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:32.200 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 159.83075778899999, 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:32.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.203 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.204 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.205 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 159.802546613, 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.207 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.208 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.209 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:32.210 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.613247245, 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:32.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.214 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.215 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.481862554, 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.216 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:32.217 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.219 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:32.220 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 155.32076515, 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:32.221 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.222 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:32.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:32.225 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 155.320712603, 2024-09-21T15:35:32.226 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.228 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.229 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.230 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.231 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 155.320664478, 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.232 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:32.234 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:32.235 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:32.236 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 155.30826759000001, 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.238 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.239 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.240 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:32.242 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.74749460800001, 2024-09-21T15:35:32.243 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:32.245 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.246 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.248 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.249 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.68047024200001, 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.250 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.251 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.252 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.253 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.545350111, 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.255 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:32.257 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.258 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.259 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.50468796800001, 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.261 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.264 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.50467475100001, 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.265 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:32.266 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:32.267 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.50466194399999, 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:32.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:32.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.50462153699999, 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:32.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.274 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:32.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.277 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.504597069, 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.278 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.279 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.280 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:32.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.485375081, 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.283 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:32.285 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.286 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 153.39807149800001, 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:32.290 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:32.291 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.292 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:32.293 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 85.403386054999999, 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:32.294 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.295 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.296 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.297 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 84.989126708000001, 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:32.299 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:32.300 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:32.301 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:32.302 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:32.303 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:32.304 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:32.302+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:32.404 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:32.402+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:32.434 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:32.434 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:32.434 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:32.434 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:32.448 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:32.505 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:32.502+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:32.571 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:32.571 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:32.571 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:32.571 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:32.571 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:32.585 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:32.606 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:32.606+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:32.707 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:32.706+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:32.713 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:32.713 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434204163999993, 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:32.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:32.715 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:32.716 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:32.717 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434180104999996, 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:32.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.719 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:32.720 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:32.721 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434156142000006, 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:32.722 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.723 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:32.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:32.725 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434132375999994, 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.726 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.727 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.728 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.729 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434108734000006, 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.730 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:32.731 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.732 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.733 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.734 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434083181000005, 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.735 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.736 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:32.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:32.738 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434038966000003, 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.739 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.740 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.741 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.742 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.434010256999997, 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.433978714000006, 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.433951641999997, 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.433925708999993, 2024-09-21T15:35:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.433390766000002, 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.432761098, 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428849534999998, 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428803036999994, 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428753868000001, 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428706020000007, 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428657630999993, 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.791 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.428576710000002, 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.793 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.794 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.795 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 85.424978347000007, 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:32.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:32.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:32.798 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:32.799 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:32.800 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:32.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:32.809 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:32.806+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:32.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:32.910 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:32.910+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:32.925 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:32.925 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:32.925 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:32.925 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:32.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:32.965 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-21T15:35:32.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T15:35:33.047 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:33.046+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:33.053 INFO:teuthology.orchestra.run.smithi107.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T15:35:33.068 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:33.069 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:33.069 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:33.069 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:33.069 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:33.071 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:33.071 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:33.083 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:33.088 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:33.106 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:33.106 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:33.110 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:33.106+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:33.123 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:33.123 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:33.141 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:33.158 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:33.158 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:33.177 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:33.177 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:33.180 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:35:33.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:35:33.205 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:33.205 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:33.211 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:33.206+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:33.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:33.312 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:33.310+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:33.387 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:33.387 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:33.401 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:33.445 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:33.442+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:33.514 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:33.510+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:33.523 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:33.523 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:33.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:33.615 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:33.614+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:33.704 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:33.704 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:33.715 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:33.710+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:33.719 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:33.816 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:33.810+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:33.872 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:33.872 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:33.884 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:33.929 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:33.926+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:33.979 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is set 2024-09-21T15:35:34.018 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:34.014+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:34.063 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.063 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:34.118 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:34.114+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:34.219 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:34.218+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:34.238 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.238 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:34.320 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:34.314+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:34.381 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.382 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.395 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:34.443 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:34.442+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:34.518 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.518 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.522 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:34.518+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:34.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:34.623 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:34.618+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:34.698 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.698 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.713 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:34.755 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:34.754+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:34.825 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:34.822+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:34.873 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:34.873 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:34.887 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:34.926 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:34.926+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:35.006 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:35.006 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:35.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:35.067 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:35.066+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:35.127 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:35.122+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:35.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:35.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:35.228 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:35.222+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:35.329 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:35.326+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:35.331 INFO:teuthology.orchestra.run.smithi107.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T15:35:35.331 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:35.331 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:35.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:35.349 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:35.349 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:35.365 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:35.366 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:35.381 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:35.382 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:35.401 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:35.422 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:35.422 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:35.430 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:35.426+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:35.442 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:35.442 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:35.455 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:35.462 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:35.468 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:35:35.468 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:35:35.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:35.555 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:35.554+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:35.623 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:35.623 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:35.623 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:35.623 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:35.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.624 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:35.624 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:35.624 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 163.253305256, 2024-09-21T15:35:35.624 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:35.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.625 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:35.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:35.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.629 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 163.22509407999999, 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.631 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.632 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:35.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.634 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 162.03579471200001, 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:35.635 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.638 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.639 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 161.90441002099999, 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:35.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:35.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:35.643 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.74331261699999, 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:35.644 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.645 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.646 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.648 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:35.649 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.74326006999999, 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:35.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.651 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.652 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.653 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.654 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:35.655 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.74321194500001, 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:35.657 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.658 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.659 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.660 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 158.730815057, 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.661 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.662 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:35.663 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.664 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.665 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.666 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 157.170042075, 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.667 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:35.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:35.669 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:35.670 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.671 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 157.103017709, 2024-09-21T15:35:35.672 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.674 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.676 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:35.677 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.96789757799999, 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.679 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.680 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.681 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.683 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.927235435, 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:35.685 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.686 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.927222218, 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.688 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:35.689 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.690 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.691 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.92720941100001, 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.692 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:35.693 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:35.694 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.695 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.92716900400001, 2024-09-21T15:35:35.696 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.698 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:35.699 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.700 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.92714453599999, 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.701 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.702 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.90792254799999, 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.706 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.707 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.708 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.709 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.710 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 156.82061896499999, 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:35.711 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.712 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.713 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.714 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.715 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.825933522, 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.716 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:35.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:35.718 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:35.719 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 88.411674175000002, 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:35.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.721 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.722 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.723 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:35.724 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:35.726 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:35.726 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:35.722+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:35.827 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:35.822+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:35.858 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:35.858 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:35.858 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:35.858 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:35.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:35.961 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:35.958+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:36.012 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T15:35:36.028 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:36.022+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:36.049 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:36.049 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:36.049 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:36.065 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:36.153 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:36.150+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:36.230 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:36.226+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.960109110999994, 2024-09-21T15:35:36.238 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:36.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:36.241 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.960085051999997, 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:36.244 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.245 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.246 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.960061089000007, 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.247 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:36.248 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.249 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.250 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:36.251 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.960037322999995, 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:36.252 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.253 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.254 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.255 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.960013681000007, 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:36.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.257 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.258 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.259 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959988128000006, 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.260 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.261 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.262 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.263 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.264 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959943913000004, 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:36.265 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.266 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.267 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.268 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959915203999998, 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:36.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.270 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.271 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.272 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959883661000006, 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:36.273 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.274 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.275 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.276 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.277 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959856588999997, 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.278 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.279 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:36.280 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.281 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959830655999994, 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.282 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.283 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:36.284 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.285 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.286 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.959295713000003, 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:36.287 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.288 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.289 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.290 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.958666045000001, 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:36.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:36.292 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.293 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954754481999998, 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:36.295 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:36.296 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.297 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.298 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.299 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954707983999995, 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:36.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:36.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.303 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954658815000002, 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.304 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.305 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:36.306 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:36.307 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954610966999994, 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:36.308 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:36.309 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.310 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.311 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954562577999994, 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.312 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.313 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:36.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:36.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.954481657000002, 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.316 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:36.317 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:36.318 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:36.319 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:36.320 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.950883293999993, 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:36.321 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.322 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.323 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.324 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.325 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.325 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:36.326 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:36.330 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:36.326+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:36.431 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:36.426+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:36.495 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:36.495 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:36.495 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:36.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:36.553 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:36.550+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:36.633 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:36.630+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:36.668 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:36.668 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:36.668 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:36.668 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:36.668 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:36.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:36.734 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:36.730+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:36.759 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is set 2024-09-21T15:35:36.834 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:36.830+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:36.864 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:36.864 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:36.864 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:36.864 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582495089999995, 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.865 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.866 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.867 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.868 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582436529000006, 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.869 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.870 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.871 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.872 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.873 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582365813999999, 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.874 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:35:36.876 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:35:36.877 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582316179000003, 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.878 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.879 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:35:36.880 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:35:36.881 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582248410999995, 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:35:36.882 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.883 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.884 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.885 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.886 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582137537999998, 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:36.887 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.888 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.889 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.890 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.582043571, 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.891 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:35:36.892 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.893 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.894 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581995445999993, 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.895 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.896 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:35:36.897 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:35:36.898 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581925768000005, 2024-09-21T15:35:36.899 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.900 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.901 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:35:36.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581876375999997, 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.906 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:35:36.907 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581829506999995, 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.908 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.909 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.910 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.911 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581782411999995, 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.912 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:36.913 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.914 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.915 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:36.916 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581733373999995, 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:35:36.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:35:36.919 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581665626000003, 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.921 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:35:36.923 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.924 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581616947000001, 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.925 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:36.926 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.927 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.928 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.929 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.581506664000003, 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.930 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.931 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.932 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:35:36.933 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.579258206999995, 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.934 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.935 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.936 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.937 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.579188455999997, 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.938 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:36.939 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.940 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.942 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.579098805000001, 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.943 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.944 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:35:36.946 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 89.578819022999994, 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:36.947 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:36.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:35:36.949 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:36.951 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:36.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:36.953 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:36.950+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:37.054 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:37.050+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:37.125 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:37.125 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:37.125 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:37.125 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:37.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:37.185 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:37.182+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:37.255 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:37.250+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:37.261 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:37.262 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:37.262 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:37.262 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:37.262 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:37.275 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:37.355 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:37.354+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:37.445 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.785215910999995, 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.446 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:37.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:35:37.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:35:37.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:37.450 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.781365328999996, 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.454 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.781340137000001, 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:37.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:35:37.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:35:37.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.777364114999997, 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.460 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.461 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:35:37.462 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.777338092999997, 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:35:37.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.464 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.465 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:35:37.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.777314540000006, 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:37.468 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:35:37.469 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.470 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.777279698000001, 2024-09-21T15:35:37.471 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.472 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.473 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.474 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:37.475 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.773298175999997, 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.476 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.477 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.479 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.773190799000005, 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.480 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:35:37.481 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.482 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.773164230000006, 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.484 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.485 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:35:37.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:35:37.487 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.773136653999998, 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:35:37.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.489 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.491 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.773113144000007, 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.492 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:37.493 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.494 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.495 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769411517999998, 2024-09-21T15:35:37.496 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.497 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.498 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:35:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.500 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769381765000006, 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.501 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.502 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.503 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.504 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769354695999994, 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.505 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.506 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.507 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.508 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769331359999995, 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.509 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:35:37.510 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.511 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769262756000003, 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.514 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:35:37.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:35:37.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.769234015999999, 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.517 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.518 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.519 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:35:37.520 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.765321033000006, 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.521 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:37.522 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:35:37.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:35:37.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 88.765265960999997, 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:37.525 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:37.526 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:37.527 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.528 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:37.529 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:37.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T15:35:37.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:37.531 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:37.530+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:37.631 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:37.626+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:37.668 INFO:teuthology.orchestra.run.smithi107.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T15:35:37.685 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:37.700 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:37.700 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:37.700 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:37.700 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:37.702 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:37.715 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:37.717 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:37.734 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:37.752 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:37.752 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:37.759 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:37.754+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:37.777 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:37.777 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:37.799 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:37.804 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:35:37.804 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:35:37.832 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:37.826+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:37.893 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:37.893 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:37.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:37.933 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:37.930+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:38.006 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 4, 5, 6, 7, 2] out_osds: [0] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-21T15:35:38.006 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T15:35:38.007 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-21T15:35:38.007 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T15:35:38.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T15:35:38.009 INFO:tasks.ceph.osd.0:Started 2024-09-21T15:35:38.009 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:38.034 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:38.030+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:38.076 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.077 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:38.135 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:38.130+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:no valid command found; 10 closest matches: 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:0 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:1 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:2 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:abort 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:assert 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:config diff 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:config diff get 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:config get 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:config help [] 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:config set ... 2024-09-21T15:35:38.136 INFO:teuthology.orchestra.run.smithi107.stderr:admin_socket: invalid command 2024-09-21T15:35:38.138 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T15:35:38.138 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-21T15:35:38.204 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.204 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:38.236 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:38.230+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:38.331 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.331 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.336 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:38.334+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:38.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:38.438 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:38.438+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:38.503 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.503 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.517 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:38.563 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:38.562+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:38.633 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.633 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.639 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:38.638+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:38.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:38.648 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:38.646+0000 7fcc1b70f7c0 -1 Falling back to public interface 2024-09-21T15:35:38.741 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:38.738+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:38.768 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.768 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:38.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:35:38.842 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:38.838+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:38.903 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:38.903 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:38.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:38.963 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:38.962+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:39.043 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:39.038+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:39.045 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.045 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:39.144 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:39.142+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:39.215 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.215 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.229 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:39.275 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:39.274+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:39.345 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:39.342+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:39.350 INFO:teuthology.orchestra.run.smithi107.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.350 INFO:teuthology.orchestra.run.smithi107.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.364 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:39.446 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:39.442+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:39.495 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.495 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:39.553 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:39.550+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:39.648 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:39.646+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:39.679 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.679 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.693 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:39.750 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:39.746+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:39.770 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:35:39.805 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T15:35:39.850 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:39.846+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:39.881 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:39.882 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:39.897 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:39.951 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:39.950+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-21T15:35:39.982 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1: {} 2024-09-21T15:35:39.982 INFO:teuthology.orchestra.run.smithi107.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T15:35:39.998 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2: {} 2024-09-21T15:35:39.998 INFO:teuthology.orchestra.run.smithi107.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T15:35:40.052 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:40.050+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:40.061 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:40.062 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:40.076 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:40.153 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:40.150+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:40.197 INFO:teuthology.orchestra.run.smithi170.stdout:ERROR: (22) Invalid argument 2024-09-21T15:35:40.197 INFO:teuthology.orchestra.run.smithi170.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T15:35:40.211 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:40.220 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3: {} 2024-09-21T15:35:40.220 INFO:teuthology.orchestra.run.smithi107.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T15:35:40.249 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4: {} 2024-09-21T15:35:40.249 INFO:teuthology.orchestra.run.smithi107.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T15:35:40.254 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:40.250+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:40.279 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5: {} 2024-09-21T15:35:40.279 INFO:teuthology.orchestra.run.smithi107.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T15:35:40.304 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6: {} 2024-09-21T15:35:40.305 INFO:teuthology.orchestra.run.smithi107.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T15:35:40.330 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7: {} 2024-09-21T15:35:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T15:35:40.335 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T15:35:40.335 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T15:35:40.355 INFO:tasks.ceph.osd.6.smithi170.stderr:2024-09-21T15:35:40.350+0000 7f6b0b41c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26380) UID: 0 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096472743999996, 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.379 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:40.380 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:35:40.381 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:35:40.382 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096414182999993, 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:35:40.383 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.384 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:35:40.385 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.386 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096343468000001, 2024-09-21T15:35:40.387 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.388 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.389 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.390 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:40.391 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096293833000004, 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:35:40.393 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:35:40.394 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096226064999996, 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:35:40.397 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.398 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096115191999999, 2024-09-21T15:35:40.400 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.401 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.404 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.096021225000001, 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:35:40.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:40.408 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095973099999995, 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.410 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.411 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095903422000006, 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:35:40.412 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:40.413 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.414 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:35:40.415 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095854029999998, 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.416 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.417 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:35:40.418 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.419 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095807160999996, 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.420 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.421 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:35:40.422 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:35:40.423 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095760065999997, 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.424 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:35:40.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:35:40.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095711027999997, 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.428 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.429 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.430 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.431 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095643280000004, 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.433 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:35:40.434 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.435 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095594601000002, 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.436 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.437 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:35:40.438 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.439 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.095484318000004, 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.440 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.443 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.093235860999997, 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.444 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.445 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.446 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.447 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.093166109999999, 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.448 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.449 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:35:40.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:35:40.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.093076459000002, 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:35:40.452 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:40.453 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:35:40.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 93.092796676999995, 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.456 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.457 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.458 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:35:40.459 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.460 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:40.461 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:40.462 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:40.458+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:40.561 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:40.558+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:40.630 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:40.631 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:40.631 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:40.631 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:40.646 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:40.662 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:40.658+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:40.763 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:40.758+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:40.771 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:40.771 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:40.771 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:40.771 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:40.771 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:40.789 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:40.877 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:40.874+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:40.965 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.306221839000003, 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.966 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.967 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.968 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.302371257000004, 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.970 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.971 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:35:40.972 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.973 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.302346064999995, 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.974 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.975 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.976 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.977 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.298370043000006, 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.978 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.979 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.980 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:35:40.981 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.298344021000005, 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.982 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.983 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.984 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.985 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.298320468, 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.986 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.987 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.988 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:35:40.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.298285625999995, 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.990 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:35:40.992 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.993 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.294304104000005, 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.994 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:40.995 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:40.996 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.997 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.294196726999999, 2024-09-21T15:35:40.998 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:40.999 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.000 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.001 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.002 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.294170158, 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.003 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.004 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.005 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:41.006 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.294142582000006, 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:35:41.008 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:35:41.009 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.010 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.294119072000001, 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.011 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.012 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.013 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.014 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290417446000006, 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:35:41.015 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.016 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:35:41.017 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:35:41.018 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290387693, 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.019 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.020 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.021 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:35:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290360624000002, 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.023 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.024 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.027 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290337288000003, 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:35:41.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:35:41.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290268683999997, 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:35:41.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:35:41.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.290239944000007, 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.036 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.037 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.286326961, 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.040 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:35:41.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 92.286271889000005, 2024-09-21T15:35:41.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.048 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:41.049 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:40.962+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:41.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:41.066 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:41.062+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:41.167 INFO:tasks.ceph.osd.7.smithi170.stderr:2024-09-21T15:35:41.162+0000 7f0c8f207640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26382) UID: 0 2024-09-21T15:35:41.217 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:41.217 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:41.217 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:41.217 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:41.231 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:41.268 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:41.266+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:41.368 INFO:tasks.ceph.osd.2.smithi107.stderr:2024-09-21T15:35:41.366+0000 7f84a14cc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26309) UID: 0 2024-09-21T15:35:41.398 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:41.398 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:41.399 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:41.399 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:41.399 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:41.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:41.469 INFO:tasks.ceph.osd.1.smithi107.stderr:2024-09-21T15:35:41.466+0000 7fe8ff29d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26307) UID: 0 2024-09-21T15:35:41.539 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:41.539 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.759954781, 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.75957679, 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:35:41.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.75935753500001, 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.548 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.758960372, 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.758556584, 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.758100822, 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:35:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.750507988, 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:35:41.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:35:41.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.749796145, 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.74653559799999, 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.572 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:35:41.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:35:41.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:35:41.575 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.745783956, 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.576 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.720856348, 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.720226692, 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:35:41.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:35:41.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.587 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.719822229, 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:35:41.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:35:41.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.591 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.719283836, 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:35:41.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.595 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.718756818, 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:35:41.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:35:41.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.599 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.71833509299999, 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:35:41.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:35:41.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.71791969100001, 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 106.717369264, 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:203828 3.b 3:d0cedbf0:::rbd_data.1191e6bca1cd.000000000000000f:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.573178731999999, 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.89398365499999999, 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 203828 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964303+0000", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964318+0000", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5262999999999999e-05 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964351+0000", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2422999999999998e-05 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964371+0000", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0432999999999999e-05 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964519+0000", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014729200000000001 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964528+0000", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9070000000000002e-06 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.854957+0000", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.89042874699999996 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.855111+0000", 2024-09-21T15:35:41.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015386600000000001 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.855241+0000", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013011899999999999 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856221+0000", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00097988799999999994 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858007+0000", 2024-09-21T15:35:41.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001786038 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858175+0000", 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016742699999999999 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858289+0000", 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114889 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.617 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:203840 3.b 3:d1028009:::rbd_data.1191e6bca1cd.000000000000008d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 23.566084604, 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.88780957400000005, 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 203840 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:41.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971398+0000", 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971415+0000", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6206e-05 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971440+0000", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5576e-05 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971481+0000", 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0757000000000003e-05 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971851+0000", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036955000000000001 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971879+0000", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8348999999999999e-05 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856332+0000", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.884453402 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:41.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856664+0000", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033178100000000001 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856983+0000", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031905200000000001 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859075+0000", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020921360000000001 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:41.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859123+0000", 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8062000000000002e-05 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859159+0000", 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5568000000000002e-05 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859210+0000", 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0618000000000002e-05 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:41.623 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:41.624 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:41.625 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T15:35:41.625 INFO:tasks.ceph.osd.3.smithi107.stderr:2024-09-21T15:35:41.622+0000 7fdd89c9c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26311) UID: 0 2024-09-21T15:35:41.725 INFO:tasks.ceph.osd.5.smithi170.stderr:2024-09-21T15:35:41.722+0000 7f4f61f5b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26378) UID: 0 2024-09-21T15:35:41.753 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:41.754 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:41.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T15:35:41.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:35:41.825 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:41.822+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:41.892 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:41.892 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:41.892 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:41.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T15:35:41.926 INFO:tasks.ceph.osd.4.smithi170.stderr:2024-09-21T15:35:41.922+0000 7f2966275640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26375) UID: 0 2024-09-21T15:35:41.929 INFO:teuthology.orchestra.run.smithi107.stdout:All operations completed A-OK! 2024-09-21T15:35:41.995 DEBUG:teuthology.parallel:result is None 2024-09-21T15:35:41.995 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T15:35:42.011 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T15:35:42.055 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T15:35:42.056 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:11975 3.0 3:007f9a71:::rbd_data.11917b2c111e.000000000000000a:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 146.922245516, 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1356111440000001, 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.057 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 11975 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123037+0000", 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.058 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123036+0000", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123043+0000", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3209999999999999e-06 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123059+0000", 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6030000000000001e-05 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.059 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123068+0000", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7809999999999994e-06 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123110+0000", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2304e-05 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:15.123118+0000", 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6109999999999996e-06 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.060 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248645+0000", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1255271840000001 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for missing object", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.248857+0000", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00021158900000000001 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252337+0000", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0034801960000000001 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.061 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.252408+0000", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1464000000000006e-05 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.253144+0000", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00073586599999999995 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.256992+0000", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0038478010000000001 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:42.062 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258528+0000", 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001535469 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258571+0000", 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.3229999999999998e-05 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:16.258648+0000", 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7454999999999996e-05 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23175 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1589248~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.480089303, 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089903758, 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23175 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:42.064 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565193+0000", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565196+0000", 2024-09-21T15:35:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.36e-06 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565204+0000", 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3229999999999998e-06 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565211+0000", 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0469999999999996e-06 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.066 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654750+0000", 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089538971999999994 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.654871+0000", 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120834 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655097+0000", 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00022582900000000001 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23176 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1871872~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.48006455700001, 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090433456999999995, 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23176 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:42.068 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565218+0000", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565217+0000", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565221+0000", 2024-09-21T15:35:42.069 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.54e-06 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565229+0000", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1869999999999997e-06 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565236+0000", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655172+0000", 2024-09-21T15:35:42.070 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089936301999999996 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655342+0000", 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000170068 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655652+0000", 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00030943700000000003 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.071 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23177 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1593344~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.480019607, 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091070499999999999, 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23177 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565263+0000", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565262+0000", 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565267+0000", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9200000000000003e-06 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565290+0000", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3306e-05 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.565298+0000", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.4660000000000002e-06 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655737+0000", 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090439344000000005 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.655926+0000", 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00018932399999999999 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.656334+0000", 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00040719600000000002 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23183 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1278976~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.476162949, 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089768261000000002, 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23183 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569120+0000", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569119+0000", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569123+0000", 2024-09-21T15:35:42.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.517e-06 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569144+0000", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1486000000000001e-05 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569154+0000", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.4369999999999996e-06 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658683+0000", 2024-09-21T15:35:42.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089529561999999993 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658768+0000", 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4594999999999999e-05 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658888+0000", 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120281 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23184 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1888256~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.476118589, 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090218647999999999, 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23184 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569164+0000", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569163+0000", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569167+0000", 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8099999999999999e-06 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569189+0000", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1849e-05 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569210+0000", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.105e-05 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.658941+0000", 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089730417000000007 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659065+0000", 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012428600000000001 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659383+0000", 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00031786299999999999 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23185 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1283072~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.083 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.47606524, 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090417352000000006, 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23185 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569217+0000", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569221+0000", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8999999999999999e-06 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569231+0000", 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0353000000000001e-05 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569241+0000", 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0017000000000001e-05 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659435+0000", 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090193837999999998 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659520+0000", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4921999999999998e-05 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659635+0000", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114992 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23186 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1892352~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.476033847, 2024-09-21T15:35:42.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090849745999999995, 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23186 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569249+0000", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569248+0000", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569252+0000", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.783e-06 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569260+0000", 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.1769999999999998e-06 2024-09-21T15:35:42.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569267+0000", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3529999999999997e-06 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659686+0000", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090419159999999998 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.659813+0000", 2024-09-21T15:35:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000126506 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660099+0000", 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028559000000000001 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23187 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1287168~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.475988121, 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091141974000000001, 2024-09-21T15:35:42.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23187 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569295+0000", 2024-09-21T15:35:42.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569294+0000", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569299+0000", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0139999999999998e-06 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569308+0000", 2024-09-21T15:35:42.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6360000000000006e-06 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569316+0000", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660147+0000", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831656999999996 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660267+0000", 2024-09-21T15:35:42.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120072 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660437+0000", 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169042 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23188 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1896448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.475958748, 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091575169999999997, 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23188 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569324+0000", 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569323+0000", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569327+0000", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7100000000000001e-06 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569335+0000", 2024-09-21T15:35:42.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4030000000000007e-06 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569342+0000", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660479+0000", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091136283999999998 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660596+0000", 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000117186 2024-09-21T15:35:42.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660899+0000", 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000303317 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23189 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1291264~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.475934188, 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091867105000000004, 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.099 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23189 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569348+0000", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569351+0000", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5039999999999998e-06 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569360+0000", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2330000000000006e-06 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569369+0000", 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.0799999999999995e-06 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.660968+0000", 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091599165999999996 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661072+0000", 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103729 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661216+0000", 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00014401 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23190 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1573888~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.47590755100001, 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.092049889999999995, 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23190 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:42.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569375+0000", 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569378+0000", 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4599999999999999e-06 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569388+0000", 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9860000000000006e-06 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.569395+0000", 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.1130000000000003e-06 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661261+0000", 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091866139999999999 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661337+0000", 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6211999999999995e-05 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.661425+0000", 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7578999999999996e-05 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23197 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1307648~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.472068984, 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089727739000000001, 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23197 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573214+0000", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573213+0000", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573216+0000", 2024-09-21T15:35:42.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1329999999999999e-06 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573228+0000", 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1297e-05 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573234+0000", 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9829999999999999e-06 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662788+0000", 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089553114000000003 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662846+0000", 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7911000000000002e-05 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662941+0000", 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5921000000000005e-05 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.111 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23198 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1590272~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.47203916399999, 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089935109999999999, 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23198 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573244+0000", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573243+0000", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.113 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573246+0000", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2499999999999998e-06 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573254+0000", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3499999999999997e-06 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573261+0000", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9070000000000001e-06 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.662980+0000", 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089718205999999995 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663059+0000", 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9753999999999996e-05 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663179+0000", 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000119263 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23199 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1868800~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.47201363400001, 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090195000999999997, 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23199 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573269+0000", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573268+0000", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573272+0000", 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4429999999999999e-06 2024-09-21T15:35:42.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573282+0000", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6569999999999992e-06 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573288+0000", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8129999999999998e-06 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663240+0000", 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089951835999999993 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663336+0000", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.6181000000000006e-05 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663464+0000", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012768100000000001 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23200 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1594368~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:42.119 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.471987928, 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090423895000000004, 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23200 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573295+0000", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573294+0000", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573298+0000", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9129999999999996e-06 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573306+0000", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4400000000000005e-06 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573313+0000", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.99e-06 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663518+0000", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090204759999999995 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663601+0000", 2024-09-21T15:35:42.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2411999999999996e-05 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663719+0000", 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118003 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23201 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1872896~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.471937888, 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090626105999999998, 2024-09-21T15:35:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23201 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573345+0000", 2024-09-21T15:35:42.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573344+0000", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573349+0000", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5299999999999998e-06 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573362+0000", 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3179999999999999e-05 2024-09-21T15:35:42.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573373+0000", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1029999999999999e-05 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663768+0000", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090395717 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663849+0000", 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0782e-05 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.663971+0000", 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012163399999999999 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23202 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1598464~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.471903088, 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090842225999999998, 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.127 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23202 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573380+0000", 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573379+0000", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573383+0000", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.67e-06 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573391+0000", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4740000000000005e-06 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573398+0000", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9059999999999996e-06 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664020+0000", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090622342999999994 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664107+0000", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6902000000000002e-05 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664222+0000", 2024-09-21T15:35:42.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000114545 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23203 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1876992~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.47187843899999, 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.091062688000000003, 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23203 2024-09-21T15:35:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573404+0000", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573403+0000", 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:42.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573407+0000", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.3670000000000001e-06 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573428+0000", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1086000000000001e-05 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.573440+0000", 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2296000000000001e-05 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664272+0000", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.090831426000000007 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664352+0000", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9955000000000003e-05 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.664467+0000", 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115401 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:23216 3.19 3:9e372b64:::rbd_data.1191f95e298.0000000000000000:18 [read 1300480~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 122.467828032, 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089893738000000001, 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 23216 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577455+0000", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577454+0000", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577457+0000", 2024-09-21T15:35:42.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.977e-06 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577465+0000", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.3769999999999996e-06 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.577472+0000", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.7900000000000002e-06 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667151+0000", 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.089678513000000001 2024-09-21T15:35:42.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667233+0000", 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2477000000000003e-05 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:33:39.667348+0000", 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000115221 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.138 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T15:35:42.255 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.255 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:42.255 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:42.255 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T15:35:42.394 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:42.394 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:42.394 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:42.394 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.410 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T15:35:42.441 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:42.438+0000 7fcc1b70f7c0 -1 osd.0 143 log_to_monitors true 2024-09-21T15:35:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.531 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:42.531 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:42.531 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:42.531 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:154354 3.14 3:2f5e897c:::rbd_data.11915dac012e.00000000000001cf:head [set-alloc-hint object_size 262144 write_size 262144,write 98304~21616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 81.214564401000004, 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83711316599999996, 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 154354 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.532 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315442+0000", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315440+0000", 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.533 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315467+0000", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7376e-05 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315479+0000", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2082999999999999e-05 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315489+0000", 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8339999999999994e-06 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.534 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315541+0000", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.1238e-05 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for peered", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:21.315547+0000", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.6370000000000002e-06 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149793+0000", 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.83424572799999996 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.535 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.149962+0000", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000169362 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150088+0000", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012603000000000001 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.150810+0000", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00072158900000000002 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:42.536 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152478+0000", 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0016683620000000001 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152518+0000", 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9665000000000001e-05 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:22.152555+0000", 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6974000000000002e-05 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197850 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8196096~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.296644321999999, 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048049983999999997, 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197850 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.538 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233362+0000", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233361+0000", 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.539 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233364+0000", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.2840000000000001e-06 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233370+0000", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4019999999999999e-06 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233375+0000", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.4320000000000001e-06 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.540 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281229+0000", 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047854053000000001 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281311+0000", 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2828e-05 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281412+0000", 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100346 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.541 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197851 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8220672~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.296610047999998, 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048216793000000001, 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197851 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:42.542 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233396+0000", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233397+0000", 2024-09-21T15:35:42.543 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5829999999999999e-06 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233401+0000", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5839999999999999e-06 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233404+0000", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9409999999999999e-06 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281448+0000", 2024-09-21T15:35:42.544 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048044188000000002 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281525+0000", 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.6911999999999998e-05 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281613+0000", 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7805000000000004e-05 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.545 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197852 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8245248~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.296599788000002, 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048430976000000001, 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197852 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233406+0000", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233407+0000", 2024-09-21T15:35:42.547 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4729999999999999e-06 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233411+0000", 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.5669999999999999e-06 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.233414+0000", 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5220000000000002e-06 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.548 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281654+0000", 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048240631999999999 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281737+0000", 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2663999999999993e-05 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.281837+0000", 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100445 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197869 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8663040~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292726694000002, 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048056726000000001, 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197869 2024-09-21T15:35:42.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237279+0000", 2024-09-21T15:35:42.551 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237280+0000", 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.125e-06 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237283+0000", 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8990000000000001e-06 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237286+0000", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6589999999999999e-06 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285233+0000", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047947417999999999 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285274+0000", 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0451999999999998e-05 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285336+0000", 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.2380999999999998e-05 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197870 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8687616~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:42.554 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292717926000002, 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048269091, 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197870 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237288+0000", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237289+0000", 2024-09-21T15:35:42.556 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0240000000000001e-06 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237292+0000", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.926e-06 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237295+0000", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.615e-06 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285371+0000", 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048076749000000002 2024-09-21T15:35:42.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285454+0000", 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2579999999999999e-05 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285557+0000", 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000103405 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.558 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197871 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8712192~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292709240000001, 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048384323999999999, 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197871 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:42.559 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237297+0000", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237298+0000", 2024-09-21T15:35:42.560 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.035e-06 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237301+0000", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237303+0000", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4990000000000001e-06 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285586+0000", 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048282486999999999 2024-09-21T15:35:42.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285627+0000", 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0973999999999998e-05 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285681+0000", 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4664999999999999e-05 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.562 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197872 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8736768~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292700793000002, 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048591882000000003, 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197872 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.563 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237305+0000", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237306+0000", 2024-09-21T15:35:42.564 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.133e-06 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237309+0000", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8590000000000001e-06 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237312+0000", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5509999999999998e-06 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285716+0000", 2024-09-21T15:35:42.565 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048404585 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285798+0000", 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2034999999999996e-05 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285897+0000", 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.8941e-05 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197873 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8761344~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292692252000002, 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048699739999999998, 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197873 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237314+0000", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237315+0000", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.113e-06 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237318+0000", 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.937e-06 2024-09-21T15:35:42.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237320+0000", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5189999999999999e-06 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285925+0000", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048604461000000002 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.285963+0000", 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8800000000000001e-05 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.573 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286014+0000", 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0117000000000001e-05 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197874 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8785920~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292683519999997, 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048909115000000003, 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.574 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197874 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237322+0000", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237323+0000", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.051e-06 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237326+0000", 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8720000000000002e-06 2024-09-21T15:35:42.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237329+0000", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6070000000000002e-06 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286049+0000", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048720326000000001 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286129+0000", 2024-09-21T15:35:42.577 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0105999999999995e-05 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286232+0000", 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102357 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197875 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8810496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:42.578 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292674585, 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049017722999999999, 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197875 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.579 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237331+0000", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237332+0000", 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1340000000000001e-06 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.580 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237335+0000", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9299999999999999e-06 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237338+0000", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5660000000000001e-06 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286259+0000", 2024-09-21T15:35:42.581 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048921521000000003 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286299+0000", 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9746000000000002e-05 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286349+0000", 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0039e-05 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197876 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 8835072~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.292625946000001, 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049188611, 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197876 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.583 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237380+0000", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237340+0000", 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999595 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.584 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237383+0000", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2809999999999998e-05 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237393+0000", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0012e-05 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.237400+0000", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.9639999999999997e-06 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.585 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286384+0000", 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048984667000000003 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286469+0000", 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.4608e-05 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.286569+0000", 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9864999999999993e-05 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197892 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9228288~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288745185000003, 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047997191000000002, 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197892 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.587 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241261+0000", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241262+0000", 2024-09-21T15:35:42.588 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2020000000000001e-06 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241265+0000", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241267+0000", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.649e-06 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289067+0000", 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.047799243999999998 2024-09-21T15:35:42.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289147+0000", 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.0631999999999996e-05 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289258+0000", 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011077 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.590 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197893 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9252864~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288736346, 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048114605999999997, 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197893 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.591 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241270+0000", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.592 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241271+0000", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3510000000000001e-06 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241274+0000", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8899999999999999e-06 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241276+0000", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6290000000000002e-06 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.593 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289292+0000", 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048015774999999997 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289334+0000", 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2283999999999997e-05 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289384+0000", 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9883000000000003e-05 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.594 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197894 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9277440~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288727485000003, 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048316051999999998, 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197894 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.595 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241279+0000", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241278+0000", 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241280+0000", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1999999999999999e-06 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241283+0000", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.9050000000000001e-06 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241285+0000", 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4930000000000001e-06 2024-09-21T15:35:42.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289419+0000", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048134254000000001 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289497+0000", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.7566999999999998e-05 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289595+0000", 2024-09-21T15:35:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.7838999999999996e-05 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197895 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9302016~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288718766000002, 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048434577999999999, 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197895 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241287+0000", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241289+0000", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3769999999999999e-06 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241291+0000", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8569999999999999e-06 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.601 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241294+0000", 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.278e-06 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289629+0000", 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048335411000000002 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289672+0000", 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2616e-05 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.602 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289722+0000", 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0250999999999997e-05 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197896 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9326592~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288710281, 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048564475000000003, 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197896 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241296+0000", 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241297+0000", 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.234e-06 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.605 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241300+0000", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8200000000000001e-06 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241302+0000", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.6079999999999998e-06 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289751+0000", 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048448349000000002 2024-09-21T15:35:42.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289806+0000", 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.5065000000000002e-05 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289860+0000", 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4605999999999999e-05 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.607 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197897 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9351168~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288701545000002, 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.04876602, 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197897 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.608 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241304+0000", 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.609 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241305+0000", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.122e-06 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241308+0000", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.869e-06 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241311+0000", 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5909999999999998e-06 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.610 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289898+0000", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048587015999999997 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.289977+0000", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9165000000000005e-05 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290071+0000", 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.3465e-05 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197898 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9375744~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288693021, 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048885534000000001, 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197898 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.612 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241313+0000", 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.613 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241314+0000", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241317+0000", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.881e-06 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241319+0000", 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.4959999999999999e-06 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.614 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290100+0000", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048781013999999998 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290142+0000", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1615000000000001e-05 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290199+0000", 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:197899 3.14 3:2f2f163a:::rbd_data.1191eefab886.000000000000000c:2a [read 9400320~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e109)", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 56.288684568999997, 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.049075287000000002, 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 197899 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:42.616 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241321+0000", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967296 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241322+0000", 2024-09-21T15:35:42.617 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.111e-06 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241325+0000", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.841e-06 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.241328+0000", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.345e-06 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290233+0000", 2024-09-21T15:35:42.618 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.048905754000000003 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290307+0000", 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3745000000000006e-05 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:46.290397+0000", 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.9697000000000005e-05 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:42.619 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.621 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T15:35:42.790 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.791 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:42.791 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:42.791 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.805 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T15:35:42.920 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:42.921 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:42.935 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T15:35:43.014 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:35:43.036 INFO:tasks.ceph.osd.0.smithi107.stderr:2024-09-21T15:35:43.034+0000 7fcc0f4d6640 -1 osd.0 161 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [ 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:613 3.b 3:d0202472:::rbd_data.11911fd826fd.00000000000000b9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 170.70271311100001, 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.042860022999999997, 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.071 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 613 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367617+0000", 2024-09-21T15:35:43.072 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367615+0000", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367629+0000", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4347e-05 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367657+0000", 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7466000000000001e-05 2024-09-21T15:35:43.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367677+0000", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0426999999999999e-05 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.367787+0000", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000109969 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.368051+0000", 2024-09-21T15:35:43.074 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026443099999999998 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.376003+0000", 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079517860000000006 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410247+0000", 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.034243863999999999 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.075 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410327+0000", 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9788000000000002e-05 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410414+0000", 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.6975e-05 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410477+0000", 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3332999999999998e-05 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.076 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop_reply(client.4497.0:613 3.b e26/21 ondisk, result = 0)", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 170.674501935, 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014664477, 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.077 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395828+0000", 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395827+0000", 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395837+0000", 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0526999999999999e-05 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.078 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395867+0000", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0303000000000001e-05 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.395887+0000", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.9293e-05 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410299+0000", 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014412813 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.079 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410314+0000", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4626e-05 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:52.410493+0000", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017864199999999999 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:1220 3.7 3:e6effa04:::rbd_data.1191bb6b225c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:43.080 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 169.48520256699999, 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.052420113999999997, 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 1220 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.081 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585128+0000", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.585124+0000", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999962 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594465+0000", 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0093415500000000005 2024-09-21T15:35:43.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594514+0000", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.9097000000000001e-05 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594533+0000", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8916e-05 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594662+0000", 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000128949 2024-09-21T15:35:43.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.594924+0000", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00026208500000000001 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.596935+0000", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.002010941 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.634051+0000", 2024-09-21T15:35:43.084 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037116017000000001 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637373+0000", 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.003321503 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637472+0000", 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.9029000000000003e-05 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.085 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.637548+0000", 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5893e-05 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:1234 3.c e27/21 3:326a944d:::rbd_data.1191bb6b225c.0000000000000003:head v 27'26, mlcod=27'23)", 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 169.35381787599999, 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.030685865, 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.086 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716512+0000", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.716511+0000", 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:43.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734468+0000", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017956551000000001 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734507+0000", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.9780000000000002e-05 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734525+0000", 2024-09-21T15:35:43.088 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.7618999999999999e-05 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734599+0000", 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.3687000000000001e-05 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.734626+0000", 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7636000000000001e-05 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.089 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747150+0000", 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012524107 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:53.747198+0000", 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7825999999999999e-05 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3166 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:43.090 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 166.19272047199999, 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012143039, 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3166 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877610+0000", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877608+0000", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877617+0000", 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.003e-06 2024-09-21T15:35:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877635+0000", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.808e-05 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877650+0000", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.435e-05 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881623+0000", 2024-09-21T15:35:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00397285 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.881840+0000", 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000217662 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.882617+0000", 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00077709999999999997 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.094 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889442+0000", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0068244239999999999 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889484+0000", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2042999999999997e-05 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889652+0000", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000168438 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.095 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889753+0000", 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000100479 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3167 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 166.19266792499999, 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01244283, 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.096 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3167 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877662+0000", 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.097 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877661+0000", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877669+0000", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8839999999999994e-06 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877685+0000", 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6288999999999999e-05 2024-09-21T15:35:43.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877699+0000", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3107e-05 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.883949+0000", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0062504420000000002 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.884181+0000", 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000232062 2024-09-21T15:35:43.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.885189+0000", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0010082450000000001 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889871+0000", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0046817769999999998 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.889919+0000", 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7673000000000002e-05 2024-09-21T15:35:43.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890001+0000", 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.2199000000000004e-05 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00010413599999999999 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3168 3.18 3:1dfa7927:::rbd_object_map.1191ab8bc3c6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 166.19261979999999, 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014231514000000001, 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3168 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.102 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877710+0000", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877709+0000", 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.103 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877717+0000", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877734+0000", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.6719999999999999e-05 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.877747+0000", 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.2843000000000001e-05 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.104 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886694+0000", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0089469110000000001 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.886947+0000", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000253718 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.888044+0000", 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001096762 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.105 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891210+0000", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0031660989999999999 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891702+0000", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00049165899999999998 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891822+0000", 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000120656 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.106 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.891942+0000", 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00011948899999999999 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3171 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:head [stat,truncate 6291456] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 166.180222912, 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011724511999999999, 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3171 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.108 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890107+0000", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890105+0000", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999976 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890119+0000", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.4219999999999999e-05 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890146+0000", 2024-09-21T15:35:43.109 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.7216000000000002e-05 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890170+0000", 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3476999999999999e-05 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890294+0000", 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012357199999999999 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.110 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890465+0000", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00017126899999999999 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 7", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.890745+0000", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00028029800000000001 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.895574+0000", 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048290859999999998 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901755+0000", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0061807709999999998 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901800+0000", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.5323e-05 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:56.901832+0000", 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.1523000000000003e-05 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.112 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3825 3.1e 3:79cec8b3:::rbd_data.119117b26f37.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29612980~29532] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.61944993, 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.053928990000000003, 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3825 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450880+0000", 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.114 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.450877+0000", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457328+0000", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0064510280000000001 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457364+0000", 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6050000000000002e-05 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.115 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457380+0000", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.5308e-05 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457512+0000", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013180599999999999 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.457910+0000", 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00039818699999999999 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.116 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.459580+0000", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.001670777 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.503999+0000", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.044418452999999997 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504525+0000", 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00052603000000000003 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504685+0000", 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00015975200000000001 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.504809+0000", 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00012452900000000001 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.118 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3898 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.552425564, 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.037314975, 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3898 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.119 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517905+0000", 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.517901+0000", 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.120 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522175+0000", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0042739010000000001 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522223+0000", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7839000000000001e-05 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522241+0000", 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.785e-05 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522320+0000", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.9357000000000006e-05 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.522546+0000", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000226314 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.523657+0000", 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0011101399999999999 2024-09-21T15:35:43.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.545483+0000", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021826186000000001 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555127+0000", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0096439909999999993 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555180+0000", 2024-09-21T15:35:43.123 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.2812999999999997e-05 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.555220+0000", 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0018000000000002e-05 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:3997 3.1e 3:7e5f0ff5:::rbd_data.119117b26f37.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.417305433, 2024-09-21T15:35:43.124 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017166388000000001, 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 3997 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653025+0000", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653024+0000", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653030+0000", 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.7300000000000002e-06 2024-09-21T15:35:43.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653040+0000", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.0387999999999999e-05 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653045+0000", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8629999999999999e-06 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653093+0000", 2024-09-21T15:35:43.127 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8594999999999998e-05 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.653212+0000", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000118188 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.655688+0000", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024764380000000001 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.669999+0000", 2024-09-21T15:35:43.128 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.014311235 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670063+0000", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.3714999999999996e-05 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670132+0000", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.8662000000000001e-05 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.670191+0000", 2024-09-21T15:35:43.129 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9456000000000001e-05 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4179 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.37664329, 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01182297, 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.130 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4179 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693687+0000", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693689+0000", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3420000000000002e-06 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693694+0000", 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.0270000000000003e-06 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693697+0000", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.2959999999999999e-06 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705326+0000", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011629146999999999 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705434+0000", 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000107811 2024-09-21T15:35:43.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705510+0000", 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5735000000000006e-05 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4180 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.376630073, 2024-09-21T15:35:43.134 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012044895, 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4180 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.135 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693700+0000", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693702+0000", 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.5730000000000002e-06 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.136 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693706+0000", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8909999999999997e-06 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693710+0000", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4620000000000001e-06 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705555+0000", 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.011845303999999999 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705657+0000", 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000102002 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705745+0000", 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.7965000000000002e-05 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.138 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4181 3.7 3:e7e6193d:::rbd_data.1191cae494be.000000000000000d:a [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.37661726600001, 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012197634000000001, 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4181 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:43.139 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693713+0000", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693712+0000", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693714+0000", 2024-09-21T15:35:43.140 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.539e-06 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693718+0000", 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7730000000000001e-06 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693723+0000", 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.3680000000000001e-06 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.141 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705778+0000", 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012055233 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705835+0000", 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.6622000000000001e-05 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705911+0000", 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.5570999999999999e-05 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4192 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.37657685900001, 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012412592, 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4192 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693753+0000", 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693755+0000", 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.8339999999999999e-06 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693759+0000", 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8859999999999997e-06 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693762+0000", 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4039999999999999e-06 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.705994+0000", 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01223219 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706073+0000", 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 7.8764999999999995e-05 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.146 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706166+0000", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.2822e-05 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4194 3.7 3:e6081e46:::rbd_data.1191ab8bc3c6.0000000000000002:9 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.37655239099999, 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012647251999999999, 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "started", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.147 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4194 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.148 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693778+0000", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693779+0000", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3829999999999999e-06 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693783+0000", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.7409999999999998e-06 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.693786+0000", 2024-09-21T15:35:43.149 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.4860000000000002e-06 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706191+0000", 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.012405111999999999 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706287+0000", 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.5441999999999999e-05 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.706425+0000", 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.00013835400000000001 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4211 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 27262976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.35733040299999, 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.039456204000000002, 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4211 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.152 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.713000+0000", 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.712996+0000", 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999967 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717881+0000", 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0048849480000000001 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.153 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717909+0000", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.8257000000000001e-05 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717921+0000", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1291000000000001e-05 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.717980+0000", 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.9475999999999998e-05 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718137+0000", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000157023 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.718702+0000", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000565064 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.734547+0000", 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015844862000000001 2024-09-21T15:35:43.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752372+0000", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.017824949 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752413+0000", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.1220999999999997e-05 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.752456+0000", 2024-09-21T15:35:43.156 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.2663999999999997e-05 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_op(client.4497.0:4244 3.18 3:1f0089c0:::rbd_data.119117b26f37.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 164.27002682, 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.018613930000000001, 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_info": { 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client": "client.4497", 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tid": 4244 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800303+0000", 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.158 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800302+0000", 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800309+0000", 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 6.4540000000000004e-06 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800322+0000", 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.3458999999999999e-05 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.159 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800331+0000", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 8.5129999999999996e-06 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800371+0000", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.0763000000000003e-05 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.800418+0000", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.6252999999999999e-05 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "waiting for subops from 0", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.802887+0000", 2024-09-21T15:35:43.160 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0024689769999999998 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818841+0000", 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.015954715000000001 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "op_commit", 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818851+0000", 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 9.1770000000000004e-06 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.161 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818887+0000", 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.6532000000000002e-05 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:32:58.818917+0000", 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.0216e-05 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:111378 3.c e79/21 3:37b8cff6:::rbd_data.119163e0d152.0000000000000001:head v 79'153, mlcod=76'151)", 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 96.275341377000004, 2024-09-21T15:35:43.162 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.021443588999999999, 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794989+0000", 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.794987+0000", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802975+0000", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0079877019999999993 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.802996+0000", 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.1024000000000001e-05 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.164 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803007+0000", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 1.1012999999999999e-05 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803061+0000", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 5.4262999999999999e-05 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.803081+0000", 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.0060000000000001e-05 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816384+0000", 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.013302333 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:06.816432+0000", 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.8831000000000001e-05 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.166 INFO:teuthology.orchestra.run.smithi107.stdout: "description": "osd_repop(client.4497.0:113454 3.15 e80/58 3:afb25ac2:::rbd_data.119163e0d152.000000000000000b:head v 80'194, mlcod=77'192)", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "initiated_at": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "age": 95.861082030000006, 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.016336872999999998, 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type_data": { 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "events": [ 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "initiated", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "throttled", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209248+0000", 2024-09-21T15:35:43.167 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "header_read", 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.209245+0000", 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "all_read", 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214515+0000", 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.0052701229999999998 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "dispatched", 2024-09-21T15:35:43.168 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214554+0000", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 3.8022999999999998e-05 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214577+0000", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 2.3025999999999999e-05 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "reached_pg", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214687+0000", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000110949 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "started", 2024-09-21T15:35:43.169 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.214735+0000", 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 4.7966999999999997e-05 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "commit_sent", 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225477+0000", 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.01074113 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "event": "done", 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "time": "2024-09-21T15:34:07.225585+0000", 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 0.000108445 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.170 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.171 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T15:35:43.171 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T15:35:43.171 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:43.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:43.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T15:35:43.304 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:43.304 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:43.305 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:43.305 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:43.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T15:35:43.340 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:43.341 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:43.341 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:43.341 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:43.441 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:43.441 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:43.441 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:43.441 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:43.441 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:43.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.574 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114108 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294883827000007, 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091388931000000007, 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114108 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277300+0000", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277299+0000", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.576 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277302+0000", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.23e-06 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277311+0000", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9369999999999993e-06 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277318+0000", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4790000000000001e-06 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368536+0000", 2024-09-21T15:35:43.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091218058000000005 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368607+0000", 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1080000000000004e-05 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368689+0000", 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1767000000000005e-05 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114109 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1402880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294859767999995, 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091643440000000007, 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114109 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277324+0000", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277323+0000", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277326+0000", 2024-09-21T15:35:43.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6169999999999998e-06 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277335+0000", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8929999999999994e-06 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277342+0000", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5169999999999996e-06 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368742+0000", 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091400332000000001 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368830+0000", 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7491999999999995e-05 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.368967+0000", 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013739899999999999 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114110 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294835805000005, 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091864425, 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114110 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:43.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277348+0000", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277347+0000", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277350+0000", 2024-09-21T15:35:43.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2430000000000001e-06 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277359+0000", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9260000000000006e-06 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277366+0000", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6100000000000002e-06 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369022+0000", 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091656557 2024-09-21T15:35:43.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369097+0000", 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4865999999999998e-05 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369212+0000", 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114836 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.586 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114111 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1423360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294812039000007, 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092090177999999995, 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114111 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:43.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277371+0000", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277374+0000", 2024-09-21T15:35:43.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2839999999999998e-06 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277383+0000", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.6859999999999995e-06 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277389+0000", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3230000000000003e-06 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369271+0000", 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091881737000000005 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369366+0000", 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.4969999999999994e-05 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369462+0000", 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.5804999999999995e-05 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114112 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1433600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.590 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294788397000005, 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092268757000000007, 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114112 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277395+0000", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277394+0000", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277397+0000", 2024-09-21T15:35:43.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0869999999999998e-06 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277406+0000", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8130000000000002e-06 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277413+0000", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6930000000000001e-06 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369516+0000", 2024-09-21T15:35:43.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092103004000000002 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369587+0000", 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0838000000000002e-05 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369664+0000", 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6945000000000003e-05 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114113 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294762844000005, 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092504049000000005, 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114113 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277421+0000", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277420+0000", 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.596 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277425+0000", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.1970000000000004e-06 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277441+0000", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6426000000000002e-05 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277453+0000", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1695999999999999e-05 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369745+0000", 2024-09-21T15:35:43.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092291877999999994 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369837+0000", 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2540999999999994e-05 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369925+0000", 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7237999999999994e-05 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114114 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1454080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294718629000002, 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092667077, 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114114 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277465+0000", 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277464+0000", 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277468+0000", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2899999999999996e-06 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277477+0000", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.3030000000000005e-06 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277484+0000", 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.369974+0000", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092490303999999995 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370044+0000", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0016000000000002e-05 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370132+0000", 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7587e-05 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.602 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114115 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294689919999996, 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092980732999999996, 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114115 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277493+0000", 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277503+0000", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0294e-05 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277512+0000", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.166e-06 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277519+0000", 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.55e-06 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370213+0000", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092693706000000001 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370382+0000", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016912900000000001 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370474+0000", 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.2517999999999997e-05 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.606 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114116 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294658377000005, 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093164668000000006, 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114116 2024-09-21T15:35:43.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277525+0000", 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277524+0000", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277528+0000", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.287e-06 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277536+0000", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.7900000000000005e-06 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.609 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277543+0000", 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4999999999999996e-06 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370522+0000", 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092978879 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370614+0000", 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.1646999999999999e-05 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370690+0000", 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.6192000000000005e-05 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114117 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1312768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294631304999996, 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093534694000000002, 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.611 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114117 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277552+0000", 2024-09-21T15:35:43.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277551+0000", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277555+0000", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5700000000000001e-06 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277564+0000", 2024-09-21T15:35:43.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.8400000000000001e-06 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277570+0000", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3670000000000002e-06 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370767+0000", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093196649000000006 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.370933+0000", 2024-09-21T15:35:43.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000165625 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371087+0000", 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015427299999999999 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114118 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1484800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294605372000007, 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093704355000000003, 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.615 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114118 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277578+0000", 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277577+0000", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277581+0000", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3759999999999999e-06 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277590+0000", 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9500000000000007e-06 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277596+0000", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2729999999999999e-06 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371135+0000", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093539132999999997 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371207+0000", 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.1347999999999995e-05 2024-09-21T15:35:43.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371282+0000", 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5897999999999997e-05 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114119 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1495040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.294070429000001, 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093572009999999997, 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.619 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114119 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278113+0000", 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.277601+0000", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994884 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278138+0000", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00053659300000000003 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278165+0000", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6820000000000001e-05 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278182+0000", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7506000000000001e-05 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371351+0000", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093168955999999997 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371521+0000", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017007 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.622 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371685+0000", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016382199999999999 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114120 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1323008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.293440760999999, 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.093151236999999998, 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.623 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114120 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278743+0000", 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.624 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278197+0000", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994545 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278749+0000", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00055213699999999996 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278764+0000", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4656e-05 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.278774+0000", 2024-09-21T15:35:43.625 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0513000000000001e-05 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371742+0000", 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092967132999999993 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371815+0000", 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3349000000000005e-05 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.626 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.371894+0000", 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.8986000000000006e-05 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114129 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289529197999997, 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091330562000000004, 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.627 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114129 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282654+0000", 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.628 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282653+0000", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282659+0000", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2360000000000001e-06 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282676+0000", 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7067000000000001e-05 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.629 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282689+0000", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2758999999999999e-05 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373847+0000", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091157949000000002 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373916+0000", 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8602999999999994e-05 2024-09-21T15:35:43.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.373985+0000", 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9218e-05 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114130 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1353728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289482699999994, 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091504663, 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114130 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282701+0000", 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.632 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282700+0000", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282706+0000", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3029999999999996e-06 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282725+0000", 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8923000000000002e-05 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.633 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282738+0000", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3063e-05 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374042+0000", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091303909000000003 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374115+0000", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3672999999999997e-05 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.634 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374205+0000", 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9884999999999997e-05 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114131 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1363968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289433531, 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091667972, 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.635 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114131 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282750+0000", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.636 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282749+0000", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282755+0000", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4729999999999997e-06 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282773+0000", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7459e-05 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.637 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282786+0000", 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3293e-05 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374265+0000", 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091479158000000005 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374343+0000", 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.7547999999999997e-05 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.638 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374418+0000", 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5143999999999997e-05 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114132 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1314816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289385683000006, 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091816461000000002, 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.639 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114132 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282798+0000", 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.640 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282797+0000", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282802+0000", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.8470000000000001e-06 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282821+0000", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8583000000000001e-05 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282835+0000", 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3956e-05 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374469+0000", 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091633724 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374542+0000", 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3128999999999997e-05 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.642 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374614+0000", 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2331999999999998e-05 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114133 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1325056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289337294000006, 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091948993000000007, 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.643 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114133 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282846+0000", 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.644 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282845+0000", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282851+0000", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0929999999999999e-06 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282868+0000", 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6739e-05 2024-09-21T15:35:43.645 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282882+0000", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3699999999999999e-05 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374660+0000", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091778085999999995 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374728+0000", 2024-09-21T15:35:43.646 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8527000000000003e-05 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374795+0000", 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6741000000000004e-05 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114134 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1374208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.289256373000001, 2024-09-21T15:35:43.647 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.092051142000000002, 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114134 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.648 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282927+0000", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282926+0000", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282933+0000", 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0600000000000002e-06 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.649 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282952+0000", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8992999999999999e-05 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.282968+0000", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6079e-05 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374839+0000", 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091871333999999999 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.650 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374908+0000", 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8649e-05 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.374978+0000", 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0454e-05 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114146 3.3 3:c01dbdfd:::rbd_data.11914be32c41.0000000000000011:1e [read 1406976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:43.651 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.285658010000006, 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091552486000000002, 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114146 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.652 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286525+0000", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286528+0000", 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3569999999999998e-06 2024-09-21T15:35:43.653 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286536+0000", 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2930000000000001e-06 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.286543+0000", 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4289999999999998e-06 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:43.654 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377831+0000", 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091288049999999996 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377945+0000", 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114596 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378078+0000", 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132588 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:43.655 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:43.656 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:43.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T15:35:43.784 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:43.784 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:43.784 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:43.784 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:43.798 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T15:35:43.918 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:43.918 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:43.919 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:43.919 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:43.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113993 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1091584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777689265000006, 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097172959000000003, 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.060 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113993 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280496+0000", 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280494+0000", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999981 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280503+0000", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2560000000000002e-06 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280521+0000", 2024-09-21T15:35:44.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8216e-05 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280544+0000", 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2725999999999999e-05 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377534+0000", 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096990171 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377601+0000", 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6923999999999998e-05 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377669+0000", 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8416000000000004e-05 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113994 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1101824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.064 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777630704000003, 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097294605000000006, 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113994 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:44.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280555+0000", 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280554+0000", 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280560+0000", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4400000000000002e-06 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280597+0000", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7088999999999997e-05 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280613+0000", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6126000000000001e-05 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377713+0000", 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097099434999999998 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377780+0000", 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7062000000000003e-05 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377849+0000", 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9380000000000003e-05 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113996 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1112064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777559988999997, 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097402632000000003, 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113996 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280625+0000", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280631+0000", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2040000000000002e-06 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280648+0000", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7549000000000001e-05 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280662+0000", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3586e-05 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377894+0000", 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097232017000000004 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.377960+0000", 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6394000000000002e-05 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378028+0000", 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7846000000000001e-05 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113997 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1122304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777510354, 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097550099000000001, 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113997 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280675+0000", 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280674+0000", 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280680+0000", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.0730000000000001e-06 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280716+0000", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5868999999999998e-05 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280730+0000", 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4042999999999999e-05 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378070+0000", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097340073999999999 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378151+0000", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0949e-05 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378225+0000", 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4058000000000001e-05 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:113999 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1132544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777442586000006, 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097689409000000005, 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 113999 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280743+0000", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280742+0000", 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280750+0000", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1999999999999994e-06 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280785+0000", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5179000000000003e-05 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280801+0000", 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5999e-05 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378275+0000", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097473911999999996 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378347+0000", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2201999999999998e-05 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378432+0000", 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4963999999999995e-05 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.080 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114000 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1142784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777331712999995, 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097765913999999995, 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114000 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.081 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280854+0000", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280852+0000", 2024-09-21T15:35:44.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280895+0000", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2852999999999998e-05 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280917+0000", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1518999999999998e-05 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280934+0000", 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7266e-05 2024-09-21T15:35:44.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378481+0000", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097547479000000006 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378552+0000", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0146000000000002e-05 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378620+0000", 2024-09-21T15:35:44.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8127999999999996e-05 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114001 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1153024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777237745999997, 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097872235000000002, 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114001 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280948+0000", 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280947+0000", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280953+0000", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.46e-06 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280970+0000", 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7362000000000001e-05 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280984+0000", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3869999999999999e-05 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378682+0000", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097698062000000002 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378751+0000", 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9052000000000002e-05 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378820+0000", 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8511999999999997e-05 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114003 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1163264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777189621000005, 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098004404000000003, 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.089 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114003 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280996+0000", 2024-09-21T15:35:44.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.280995+0000", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281000+0000", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.5890000000000002e-06 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281038+0000", 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7752e-05 2024-09-21T15:35:44.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281052+0000", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4100000000000001e-05 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378865+0000", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097812494999999999 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.378931+0000", 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5947999999999999e-05 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379000+0000", 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9505999999999995e-05 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114004 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1173504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777119943000002, 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098118762999999998, 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.093 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114004 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281066+0000", 2024-09-21T15:35:44.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281064+0000", 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281073+0000", 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5490000000000006e-06 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281090+0000", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.719e-05 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281104+0000", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3506000000000001e-05 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379048+0000", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097944328999999997 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379117+0000", 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9085999999999996e-05 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379184+0000", 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7305999999999996e-05 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114005 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1183744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777070550999994, 2024-09-21T15:35:44.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098249536999999998, 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114005 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281115+0000", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281114+0000", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281120+0000", 2024-09-21T15:35:44.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.604e-06 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281136+0000", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6818999999999999e-05 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281150+0000", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.366e-05 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379229+0000", 2024-09-21T15:35:44.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098078987000000006 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379295+0000", 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6068000000000005e-05 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379364+0000", 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9315999999999998e-05 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114006 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1193984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.777023682000006, 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098399206000000003, 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114006 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.102 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281162+0000", 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281161+0000", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281167+0000", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.9529999999999996e-06 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281184+0000", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6917000000000001e-05 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281198+0000", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3856000000000001e-05 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379410+0000", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098212397000000007 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379490+0000", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0050999999999996e-05 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379561+0000", 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0986000000000001e-05 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114007 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.776976587000007, 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098542029000000003, 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.106 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114007 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281209+0000", 2024-09-21T15:35:44.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281208+0000", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281213+0000", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.3170000000000001e-06 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281233+0000", 2024-09-21T15:35:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9945999999999998e-05 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281247+0000", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3523000000000001e-05 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379609+0000", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098361916999999993 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379682+0000", 2024-09-21T15:35:44.109 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2857999999999999e-05 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379751+0000", 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9342000000000001e-05 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114009 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1214464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:44.110 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.776927549000007, 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098672750000000004, 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114009 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.111 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281258+0000", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281257+0000", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281263+0000", 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.7130000000000002e-06 2024-09-21T15:35:44.112 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281281+0000", 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8057e-05 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281313+0000", 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2361999999999999e-05 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.113 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379796+0000", 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098482754000000006 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379863+0000", 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7119999999999994e-05 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379931+0000", 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7683999999999998e-05 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.114 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114010 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1224704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.776859801000001, 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098789717999999999, 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114010 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.115 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281326+0000", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281325+0000", 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.116 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281331+0000", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8820000000000003e-06 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281349+0000", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7677e-05 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281362+0000", 2024-09-21T15:35:44.117 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2911999999999999e-05 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.379975+0000", 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098613477000000005 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380043+0000", 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7113999999999995e-05 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.118 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380115+0000", 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2792000000000004e-05 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114012 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1234944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.776811121999998, 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098924870999999998, 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.119 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114012 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281374+0000", 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.120 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281373+0000", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281379+0000", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.677e-06 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281396+0000", 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6728999999999999e-05 2024-09-21T15:35:44.121 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281411+0000", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5123000000000001e-05 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380162+0000", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098751487999999998 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380230+0000", 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7162000000000005e-05 2024-09-21T15:35:44.122 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380299+0000", 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9626e-05 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114013 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1245184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.776700839, 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098995428999999996, 2024-09-21T15:35:44.123 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114013 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.124 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281485+0000", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281483+0000", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281490+0000", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8929999999999999e-06 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.125 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281517+0000", 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6942999999999999e-05 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.281532+0000", 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4729000000000001e-05 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380342+0000", 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.098809938999999999 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380411+0000", 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8541999999999995e-05 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380480+0000", 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9510000000000004e-05 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.127 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114016 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1265664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.774452382000007, 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097115004000000005, 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114016 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.128 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283733+0000", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283732+0000", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283752+0000", 2024-09-21T15:35:44.129 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0032999999999999e-05 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283770+0000", 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8393000000000002e-05 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283785+0000", 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4963000000000001e-05 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.130 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380711+0000", 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096926188999999996 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380778+0000", 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7092000000000001e-05 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380848+0000", 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9896999999999998e-05 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114018 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1275904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.774382630999995, 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097223041999999996, 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114018 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.132 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283803+0000", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283802+0000", 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.133 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283808+0000", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.3729999999999998e-06 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283826+0000", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7453999999999999e-05 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283863+0000", 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.7737999999999997e-05 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380890+0000", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097026846 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.380956+0000", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6042000000000002e-05 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381026+0000", 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9602000000000002e-05 2024-09-21T15:35:44.135 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114019 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1286144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.774292979999998, 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097307472000000006, 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114019 2024-09-21T15:35:44.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283892+0000", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283891+0000", 2024-09-21T15:35:44.137 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283899+0000", 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.3470000000000001e-06 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283936+0000", 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.6992000000000001e-05 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.138 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283951+0000", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5113000000000001e-05 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381068+0000", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097117142000000004 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381134+0000", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.5755999999999999e-05 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.139 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381200+0000", 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6361999999999999e-05 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:114020 3.16 3:6c0719dd:::rbd_data.11914be32c41.000000000000000f:1e [read 1296384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 96.774013198000006, 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.097203908000000006, 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.140 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 114020 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284172+0000", 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.141 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.283977+0000", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999805 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284183+0000", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020547100000000001 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284272+0000", 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.9227000000000005e-05 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.284325+0000", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.2898000000000001e-05 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381244+0000", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.096919138000000002 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381310+0000", 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6211999999999996e-05 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.143 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:07.381376+0000", 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.601e-05 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.144 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.145 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-21T15:35:44.268 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.268 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:44.269 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:44.269 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-21T15:35:44.399 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.399 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:44.399 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:44.399 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:44.399 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-21T15:35:44.539 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:44.539 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125613 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1918976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.878413355000006, 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089560267999999998, 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125613 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.540 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658713+0000", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658712+0000", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.541 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658716+0000", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3060000000000002e-06 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658724+0000", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3389999999999993e-06 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.658731+0000", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9040000000000003e-06 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.542 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748005+0000", 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089273889999999995 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748122+0000", 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011682 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.748273+0000", 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015163199999999999 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125625 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1959936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.874562772999994, 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089874489000000002, 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125625 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.544 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662564+0000", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662563+0000", 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662566+0000", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2260000000000001e-06 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662574+0000", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2099999999999993e-06 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662583+0000", 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.4130000000000006e-06 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.546 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752160+0000", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089576694999999998 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752281+0000", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012142 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752438+0000", 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015719200000000001 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.547 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125626 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1880064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.874537580999998, 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090184474000000001, 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125626 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.548 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662589+0000", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662588+0000", 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.549 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662592+0000", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2600000000000001e-06 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662602+0000", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0107e-05 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.662608+0000", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8759999999999999e-06 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.550 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752505+0000", 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089896949000000004 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752625+0000", 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120043 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.752773+0000", 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147856 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.551 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125636 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.870561558999995, 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089699066999999993, 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125636 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.552 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666565+0000", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666564+0000", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.553 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666567+0000", 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.236e-06 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666576+0000", 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1429999999999998e-06 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666582+0000", 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7530000000000004e-06 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.554 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.755987+0000", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.08940505 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756107+0000", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119537 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756264+0000", 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015700100000000001 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.555 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125637 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.870535536999995, 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090020849, 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125637 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.556 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666591+0000", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666590+0000", 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666594+0000", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.63e-06 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666602+0000", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2039999999999997e-06 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666609+0000", 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8789999999999997e-06 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.558 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756335+0000", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089726730000000005 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756454+0000", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011889 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756612+0000", 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015747600000000001 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.559 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125638 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2011136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.870511984000004, 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090337591999999994, 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125638 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.560 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666614+0000", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666617+0000", 2024-09-21T15:35:44.561 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.0900000000000001e-06 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666627+0000", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0117e-05 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666643+0000", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5849e-05 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756682+0000", 2024-09-21T15:35:44.562 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090038828000000001 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756800+0000", 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118523 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.756952+0000", 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015178199999999999 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.563 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125639 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.870477141999999, 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090643616999999996, 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125639 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666649+0000", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666648+0000", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.565 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666652+0000", 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5729999999999999e-06 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666660+0000", 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3159999999999997e-06 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.666679+0000", 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8470000000000001e-05 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.566 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757020+0000", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090341365000000007 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757139+0000", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011896300000000001 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.757293+0000", 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015375600000000001 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125648 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2052096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.866495619999995, 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089974196000000006, 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125648 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.568 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670631+0000", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670630+0000", 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.569 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670633+0000", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670642+0000", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2870000000000004e-06 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670668+0000", 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6308999999999998e-05 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760278+0000", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089610026999999995 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760424+0000", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145542 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760605+0000", 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018149799999999999 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.571 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125649 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2023424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.866388243000003, 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090235175000000001, 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125649 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.572 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670738+0000", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670737+0000", 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.573 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670741+0000", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.9999999999999998e-06 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670751+0000", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0006e-05 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670759+0000", 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.4370000000000001e-06 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.574 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760678+0000", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089918867 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760812+0000", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013459299999999999 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.760973+0000", 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016122200000000001 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.575 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125650 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2062336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.866361674000004, 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090523690000000004, 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125650 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.576 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670765+0000", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670764+0000", 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.577 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670768+0000", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3900000000000002e-06 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670779+0000", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.152e-05 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670786+0000", 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8059999999999997e-06 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.578 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761019+0000", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090232822000000004 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761136+0000", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000117269 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761288+0000", 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015250600000000001 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125651 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.866334097999996, 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090804665000000007, 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125651 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670792+0000", 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.581 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670795+0000", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5829999999999998e-06 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670803+0000", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.1629999999999996e-06 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670810+0000", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6839999999999999e-06 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.582 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761331+0000", 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090520586 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761449+0000", 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761597+0000", 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014786899999999999 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.583 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125652 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.866310588000005, 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091090018999999994, 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125652 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.584 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670816+0000", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670815+0000", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670818+0000", 2024-09-21T15:35:44.585 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.14e-06 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670827+0000", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2029999999999992e-06 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.670833+0000", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8399999999999997e-06 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761644+0000", 2024-09-21T15:35:44.586 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090810663999999999 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761760+0000", 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000116047 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.761906+0000", 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014572899999999999 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.587 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125659 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862608961999996, 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089677925000000006, 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125659 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:44.588 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674517+0000", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674520+0000", 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.033e-06 2024-09-21T15:35:44.589 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674534+0000", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4279e-05 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674541+0000", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7800000000000003e-06 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.763926+0000", 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089384833999999996 2024-09-21T15:35:44.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764044+0000", 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000118476 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764195+0000", 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015113600000000001 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125660 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 2093056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.591 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862579209000003, 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089951073000000006, 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125660 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.592 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674547+0000", 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.593 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674550+0000", 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.54e-06 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674562+0000", 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1736000000000001e-05 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.594 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674569+0000", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8800000000000002e-06 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764238+0000", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089668900999999995 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764351+0000", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000113071 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.595 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764498+0000", 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014756200000000001 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125661 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1841152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862552140000005, 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090230608000000004, 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.596 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125661 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.597 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674574+0000", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674577+0000", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.3830000000000001e-06 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674585+0000", 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2339999999999994e-06 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.598 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674592+0000", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6789999999999999e-06 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764540+0000", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089948417000000003 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764653+0000", 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011259 2024-09-21T15:35:44.599 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764805+0000", 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015191200000000001 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125662 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1851392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862528803999993, 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090505633000000002, 2024-09-21T15:35:44.600 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125662 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.601 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674598+0000", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674597+0000", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999995 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674600+0000", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.207e-06 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674609+0000", 2024-09-21T15:35:44.602 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.3529999999999995e-06 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674619+0000", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.0030000000000001e-05 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764846+0000", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090227777999999995 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.764959+0000", 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000112553 2024-09-21T15:35:44.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765103+0000", 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000144299 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125663 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1861632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:44.604 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862460200000001, 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090844069999999999, 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125663 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:44.605 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674666+0000", 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674665+0000", 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674669+0000", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.8630000000000001e-06 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674681+0000", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.136e-05 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674688+0000", 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5499999999999997e-06 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.607 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765183+0000", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090494839999999993 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765329+0000", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014602899999999999 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765510+0000", 2024-09-21T15:35:44.608 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018138400000000001 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125664 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1871872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.862431459999996, 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.091142610999999998, 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.609 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125664 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674695+0000", 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.610 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674694+0000", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674698+0000", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.659e-06 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674710+0000", 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2227e-05 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.611 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.674717+0000", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.956e-06 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765563+0000", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.090845694000000005 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765683+0000", 2024-09-21T15:35:44.612 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000120139 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.765838+0000", 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000154826 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125672 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1943552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.613 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.858518477000004, 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089678298000000004, 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125672 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678608+0000", 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678607+0000", 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678613+0000", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.2199999999999997e-06 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678637+0000", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.4093000000000001e-05 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678651+0000", 2024-09-21T15:35:44.616 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4015999999999999e-05 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.767961+0000", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089309569000000005 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768106+0000", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000145329 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768286+0000", 2024-09-21T15:35:44.617 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000180201 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:125673 3.a 3:53b6f507:::rbd_data.11914be32c41.0000000000000003:1e [read 1953792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 95.858463404999995, 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089949123000000006, 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:44.618 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 125673 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678663+0000", 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.619 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678662+0000", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.999999 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678669+0000", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9430000000000003e-06 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678682+0000", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2969000000000001e-05 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:44.620 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.678689+0000", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5229999999999998e-06 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768335+0000", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.089645453 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768453+0000", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00011859899999999999 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:34:08.768612+0000", 2024-09-21T15:35:44.621 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015870200000000001 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:44.622 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.623 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-21T15:35:44.750 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.750 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:44.750 INFO:teuthology.orchestra.run.smithi170.stdout: "num_ops": 0 2024-09-21T15:35:44.750 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-21T15:35:44.886 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:44.886 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [], 2024-09-21T15:35:44.886 INFO:teuthology.orchestra.run.smithi170.stdout: "complaint_time": 30, 2024-09-21T15:35:44.886 INFO:teuthology.orchestra.run.smithi170.stdout: "num_blocked_ops": 0 2024-09-21T15:35:44.886 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:44.900 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout: "size": 20, 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 600, 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout: "ops": [ 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.024 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95527 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9659392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.244649127, 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10485481000000001, 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95527 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.025 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777530+0000", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777118+0000", 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995885 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.026 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777534+0000", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041555999999999999 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777545+0000", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.1333e-05 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777552+0000", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.0180000000000004e-06 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.027 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.881849+0000", 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104297501 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882091+0000", 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000241647 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882385+0000", 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029349699999999999 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95528 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8631296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.24427113599999, 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492957, 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95528 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.029 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777908+0000", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777559+0000", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996514 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.030 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777911+0000", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00035185700000000002 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777919+0000", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.5869999999999993e-06 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777926+0000", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.6309999999999997e-06 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882516+0000", 2024-09-21T15:35:45.031 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104589523 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882667+0000", 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015162899999999999 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882837+0000", 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00017012700000000001 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95529 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9663488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.244051881, 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10508118900000001, 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95529 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.033 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778127+0000", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.777938+0000", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9998107 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.034 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778131+0000", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00019318199999999999 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778138+0000", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.5310000000000004e-06 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778145+0000", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.4479999999999999e-06 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.882928+0000", 2024-09-21T15:35:45.035 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104782927 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883053+0000", 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000125122 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883208+0000", 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015536 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.036 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95530 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8635392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.243654718, 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.105013738, 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95530 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.037 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778524+0000", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778190+0000", 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996657 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.038 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778527+0000", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000337182 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778540+0000", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.313e-05 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778548+0000", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 7.2989999999999999e-06 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.039 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883265+0000", 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10471763000000001 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883389+0000", 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00012351500000000001 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883538+0000", 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014910600000000001 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.040 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95531 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9667584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.24325093, 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10492913, 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95531 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778928+0000", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778553+0000", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996257 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778931+0000", 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00037741400000000002 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778944+0000", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.3468999999999999e-05 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778951+0000", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8519999999999998e-06 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883584+0000", 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10463320600000001 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.043 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883709+0000", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000124838 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883857+0000", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014787700000000001 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95532 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8639488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:45.044 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.242795168, 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10479161100000001, 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95532 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.045 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779384+0000", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.778970+0000", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995866 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779387+0000", 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00041665300000000001 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.046 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779395+0000", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.0260000000000007e-06 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.779401+0000", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.669e-06 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.883908+0000", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104506759 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.047 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884027+0000", 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000119288 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.884175+0000", 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000147847 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95553 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9712640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.23520233399999, 2024-09-21T15:35:45.048 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104746935, 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95553 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.049 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786977+0000", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786644+0000", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996676 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.786985+0000", 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00034075000000000002 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.050 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787000+0000", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4764e-05 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787014+0000", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4425e-05 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891430+0000", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104415491 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.051 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891562+0000", 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000132447 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.891723+0000", 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016129299999999999 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95555 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9716736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.234490491, 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10474705500000001, 2024-09-21T15:35:45.052 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95555 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787688+0000", 2024-09-21T15:35:45.053 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787393+0000", 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997044 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787691+0000", 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00029846400000000002 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787704+0000", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.2807e-05 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.787711+0000", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.7020000000000004e-06 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892155+0000", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443754 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892283+0000", 2024-09-21T15:35:45.055 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000128138 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.892435+0000", 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015278000000000001 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95564 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8705024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.23122994400001, 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10482190199999999, 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.056 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95564 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.057 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790949+0000", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790648+0000", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996986 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790952+0000", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000304708 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.058 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790969+0000", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6113999999999999e-05 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.790975+0000", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.9399999999999996e-06 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895431+0000", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10445505200000001 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895590+0000", 2024-09-21T15:35:45.059 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015919999999999999 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.895771+0000", 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00018109499999999999 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95566 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8709120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.23047830199999, 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10481760499999999, 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.060 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95566 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791701+0000", 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.061 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791425+0000", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997249 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791703+0000", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027804300000000001 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791712+0000", 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 8.2460000000000003e-06 2024-09-21T15:35:45.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.791718+0000", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 6.8600000000000004e-06 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896212+0000", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10449367399999999 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896343+0000", 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013063500000000001 2024-09-21T15:35:45.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.896518+0000", 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000175422 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95622 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9856000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.205550694, 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104825182, 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.064 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95622 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816628+0000", 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816332+0000", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9997039 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816638+0000", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000305624 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816659+0000", 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.1602000000000001e-05 2024-09-21T15:35:45.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816676+0000", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6385999999999999e-05 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.920940+0000", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104264623 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921181+0000", 2024-09-21T15:35:45.067 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00024107600000000001 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921453+0000", 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027203200000000001 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95623 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8823808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.20492103799999, 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10475738, 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.068 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95623 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817258+0000", 2024-09-21T15:35:45.069 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.816689+0000", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9994307 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817281+0000", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00059189500000000005 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817307+0000", 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6656e-05 2024-09-21T15:35:45.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817337+0000", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.9660999999999999e-05 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921578+0000", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104240844 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.921779+0000", 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00020110199999999999 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922015+0000", 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000236331 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95624 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9860096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.204516575, 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104989495, 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.072 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95624 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817662+0000", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.073 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817356+0000", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996939 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817672+0000", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031573899999999999 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817694+0000", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2413e-05 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.074 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817713+0000", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.8984999999999999e-05 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922116+0000", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104402121 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922376+0000", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00026041200000000001 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.075 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922652+0000", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00027583499999999999 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95625 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8827904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.203978182, 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104809106, 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.076 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95625 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818201+0000", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.817751+0000", 2024-09-21T15:35:45.077 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995508 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818211+0000", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00045919999999999999 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818253+0000", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.2082000000000001e-05 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818273+0000", 2024-09-21T15:35:45.078 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0775999999999999e-05 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922728+0000", 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104454252 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.922868+0000", 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014049000000000001 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.079 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923010+0000", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014161299999999999 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95626 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9864192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.203451164, 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10491020500000001, 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.080 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95626 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818728+0000", 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.081 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818312+0000", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995847 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818752+0000", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00043994699999999999 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818768+0000", 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6186000000000001e-05 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.082 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818783+0000", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.4382999999999999e-05 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923106+0000", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10432356700000001 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923355+0000", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000248323 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.083 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923638+0000", 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00028334399999999999 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95627 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8832000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.20302943900001, 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104843776, 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.084 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95627 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819149+0000", 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.818792+0000", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996424 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819171+0000", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000379544 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819199+0000", 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.7053e-05 2024-09-21T15:35:45.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819221+0000", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.2101999999999999e-05 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923712+0000", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104490969 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923818+0000", 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000106891 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.923993+0000", 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000174752 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95628 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 9868288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.202614037, 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104821187, 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.088 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95628 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819565+0000", 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819239+0000", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9996743 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819574+0000", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033473299999999998 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819599+0000", 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5045000000000002e-05 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.090 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819618+0000", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.9505999999999999e-05 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924062+0000", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.104443758 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924218+0000", 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015554100000000001 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.091 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924386+0000", 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016847300000000001 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:95629 3.e 3:7614ea6a:::rbd_data.1191f95e298.0000000000000004:18 [read 8836096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 110.20206361, 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10480202499999999, 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "started", 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.092 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 95629 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820115+0000", 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.819658+0000", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9995427 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820125+0000", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00046760299999999997 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820151+0000", 2024-09-21T15:35:45.094 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.6021999999999999e-05 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.820169+0000", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.7163000000000002e-05 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924473+0000", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.10430392500000001 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924675+0000", 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000202661 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:33:54.924917+0000", 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000242138 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:203828 3.b 3:d0cedbf0:::rbd_data.1191e6bca1cd.000000000000000f:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.057873078, 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.89398365499999999, 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.096 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 203828 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964306+0000", 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.097 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964303+0000", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999971 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964318+0000", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.5262999999999999e-05 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964351+0000", 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.2422999999999998e-05 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.098 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964371+0000", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.0432999999999999e-05 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964519+0000", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00014729200000000001 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.964528+0000", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 9.9070000000000002e-06 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.099 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.854957+0000", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.89042874699999996 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.855111+0000", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00015386600000000001 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.855241+0000", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00013011899999999999 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856221+0000", 2024-09-21T15:35:45.100 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00097988799999999994 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858007+0000", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.001786038 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858175+0000", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00016742699999999999 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.858289+0000", 2024-09-21T15:35:45.101 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.000114889 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "description": "osd_op(client.4497.0:203840 3.b 3:d1028009:::rbd_data.1191e6bca1cd.000000000000008d:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "initiated_at": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "age": 27.050778950000002, 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.88780957400000005, 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "type_data": { 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_info": { 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client": "client.4497", 2024-09-21T15:35:45.102 INFO:teuthology.orchestra.run.smithi170.stdout: "client_addr": "172.21.15.107:0/2023827102", 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "tid": 203840 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "events": [ 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "initiated", 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "throttled", 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971400+0000", 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.103 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "header_read", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971398+0000", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4294967295.9999986 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "all_read", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971415+0000", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 1.6206e-05 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "dispatched", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971440+0000", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.5576e-05 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "queued_for_pg", 2024-09-21T15:35:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971481+0000", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.0757000000000003e-05 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971851+0000", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00036955000000000001 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for peered", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:17.971879+0000", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 2.8348999999999999e-05 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "reached_pg", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856332+0000", 2024-09-21T15:35:45.105 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.884453402 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "started", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856664+0000", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00033178100000000001 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "waiting for subops from 6", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.856983+0000", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.00031905200000000001 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "op_commit", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859075+0000", 2024-09-21T15:35:45.106 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 0.0020921360000000001 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "sub_op_commit_rec", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859123+0000", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 4.8062000000000002e-05 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "commit_sent", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859159+0000", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 3.5568000000000002e-05 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: { 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "event": "done", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "time": "2024-09-21T15:35:18.859210+0000", 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: "duration": 5.0618000000000002e-05 2024-09-21T15:35:45.107 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-09-21T15:35:45.108 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-09-21T15:35:45.108 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-09-21T15:35:45.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T15:35:45.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T15:35:45.243 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:45.243 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:45.243 INFO:teuthology.orchestra.run.smithi107.stdout: "num_ops": 0 2024-09-21T15:35:45.243 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:45.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T15:35:45.373 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:45.373 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [], 2024-09-21T15:35:45.373 INFO:teuthology.orchestra.run.smithi107.stdout: "complaint_time": 30, 2024-09-21T15:35:45.374 INFO:teuthology.orchestra.run.smithi107.stdout: "num_blocked_ops": 0 2024-09-21T15:35:45.374 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:45.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T15:35:45.505 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:35:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: "size": 20, 2024-09-21T15:35:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: "duration": 600, 2024-09-21T15:35:45.505 INFO:teuthology.orchestra.run.smithi107.stdout: "ops": [] 2024-09-21T15:35:45.505 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:35:46.037 INFO:teuthology.orchestra.run.smithi107.stderr:noscrub is unset 2024-09-21T15:35:46.062 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T15:35:46.763 INFO:teuthology.orchestra.run.smithi107.stderr:nodeep-scrub is unset 2024-09-21T15:35:48.355 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-21T15:35:48.356 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-21T15:35:48.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-21T15:35:48.666 INFO:tasks.ceph.mon.a.smithi107.stderr:2024-09-21T15:35:48.662+0000 7fbc162d8640 -1 mon.a@0(leader).osd e164 definitely_dead 0 2024-09-21T15:35:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:marked in osd.0. 2024-09-21T15:35:49.072 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-21T15:35:49.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T15:35:49.203 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a: {} 2024-09-21T15:35:49.203 INFO:teuthology.orchestra.run.smithi107.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:35:49.216 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b: {} 2024-09-21T15:35:49.216 INFO:teuthology.orchestra.run.smithi107.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:35:49.225 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c: {} 2024-09-21T15:35:49.226 INFO:teuthology.orchestra.run.smithi107.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T15:35:49.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T15:35:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T15:35:50.083 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T15:35:51.066 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T15:35:51.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T15:35:51.770 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T15:35:51.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T15:35:52.769 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T15:35:52.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T15:35:53.771 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T15:35:53.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T15:35:54.773 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T15:35:54.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T15:35:55.776 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T15:35:55.806 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T15:35:57.117 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T15:35:57.145 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T15:35:58.117 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T15:35:58.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T15:35:59.128 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T15:35:59.156 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-21T15:36:00.138 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T15:36:00.161 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-21T15:36:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T15:36:01.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-21T15:36:02.170 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T15:36:02.192 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-21T15:36:03.170 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T15:36:03.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T15:36:04.192 INFO:teuthology.orchestra.run.smithi107.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T15:36:04.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T15:36:05.194 INFO:teuthology.orchestra.run.smithi107.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T15:36:05.219 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T15:36:05.219 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T15:36:05.219 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T15:36:05.219 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T15:36:05.219 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T15:36:05.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:36:05.514 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:05.514 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":181,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:36:05.186948+0000","last_up_change":"2024-09-21T15:35:44.012702+0000","last_in_change":"2024-09-21T15:35:49.035022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T15:32:35.528036+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T15:32:44.899492+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":160,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":143,"up_from":162,"up_thru":171,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6825","nonce":997327856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6827","nonce":997327856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6831","nonce":997327856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6829","nonce":997327856}]},"public_addr":"172.21.15.107:6825/997327856","cluster_addr":"172.21.15.107:6827/997327856","heartbeat_back_addr":"172.21.15.107:6831/997327856","heartbeat_front_addr":"172.21.15.107:6829/997327856","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":144},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[7,6]},{"pgid":"3.14","osds":[2,6]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:36:05.534 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T15:36:05.534 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:36:05.829 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:05.829 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":181,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:36:05.186948+0000","last_up_change":"2024-09-21T15:35:44.012702+0000","last_in_change":"2024-09-21T15:35:49.035022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T15:32:35.528036+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T15:32:44.899492+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":160,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":143,"up_from":162,"up_thru":171,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6825","nonce":997327856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6827","nonce":997327856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6831","nonce":997327856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6829","nonce":997327856}]},"public_addr":"172.21.15.107:6825/997327856","cluster_addr":"172.21.15.107:6827/997327856","heartbeat_back_addr":"172.21.15.107:6831/997327856","heartbeat_front_addr":"172.21.15.107:6829/997327856","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":144},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[7,6]},{"pgid":"3.14","osds":[2,6]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:36:05.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T15:36:05.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T15:36:05.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T15:36:05.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T15:36:05.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T15:36:05.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T15:36:05.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T15:36:05.852 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T15:36:06.101 INFO:teuthology.orchestra.run.smithi107.stdout:55834574896 2024-09-21T15:36:06.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:36:06.103 INFO:teuthology.orchestra.run.smithi107.stdout:695784701959 2024-09-21T15:36:06.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:36:06.115 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:36:06.117 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:36:06.119 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.119 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:36:06.123 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.124 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:36:06.126 INFO:teuthology.orchestra.run.smithi107.stdout:55834574896 2024-09-21T15:36:06.126 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:36:06.130 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:36:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.640 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:06.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.5 2024-09-21T15:36:06.659 INFO:teuthology.orchestra.run.smithi107.stdout:695784701958 2024-09-21T15:36:06.660 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:06.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.1 2024-09-21T15:36:06.667 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:06.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.3 2024-09-21T15:36:06.681 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:06.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.2 2024-09-21T15:36:06.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 695784701959 got 695784701958 for osd.0 2024-09-21T15:36:06.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.4 2024-09-21T15:36:06.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.6 2024-09-21T15:36:06.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.7 2024-09-21T15:36:07.658 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:36:07.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T15:36:07.674 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:36:07.695 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T15:36:07.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T15:36:07.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T15:36:07.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T15:36:07.707 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T15:36:08.132 INFO:teuthology.orchestra.run.smithi107.stdout:55834574894 2024-09-21T15:36:08.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574894 for osd.5 2024-09-21T15:36:08.201 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:08.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574895 for osd.3 2024-09-21T15:36:08.233 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:08.235 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:08.237 INFO:teuthology.orchestra.run.smithi107.stdout:55834574896 2024-09-21T15:36:08.250 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:08.257 INFO:teuthology.orchestra.run.smithi107.stdout:695784701959 2024-09-21T15:36:08.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.2 2024-09-21T15:36:08.260 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:08.262 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:08.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.4 2024-09-21T15:36:08.268 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:08.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574896 for osd.1 2024-09-21T15:36:08.269 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:08.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.6 2024-09-21T15:36:08.277 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:08.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 695784701959 got 695784701959 for osd.0 2024-09-21T15:36:08.287 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:08.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.7 2024-09-21T15:36:08.289 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:09.156 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T15:36:09.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T15:36:09.452 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-21T15:36:09.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574895 got 55834574895 for osd.5 2024-09-21T15:36:09.471 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:09.514 INFO:teuthology.orchestra.run.smithi107.stdout:55834574896 2024-09-21T15:36:09.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574896 got 55834574896 for osd.3 2024-09-21T15:36:09.531 DEBUG:teuthology.parallel:result is None 2024-09-21T15:36:09.531 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T15:36:09.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:09.808 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:09.809 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:09.825 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T15:36:09.694409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.526897"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5569,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551934+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:04.551934+0000","last_peered":"2024-09-21T15:36:04.551934+0000","last_clean":"2024-09-21T15:36:04.551934+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:04.551934+0000","last_undegraded":"2024-09-21T15:36:04.551934+0000","last_fullsized":"2024-09-21T15:36:04.551934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:12.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10066,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551953+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:04.551953+0000","last_peered":"2024-09-21T15:36:04.551953+0000","last_clean":"2024-09-21T15:36:04.551953+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:04.551953+0000","last_undegraded":"2024-09-21T15:36:04.551953+0000","last_fullsized":"2024-09-21T15:36:04.551953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:06.582849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33607,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218602+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.218602+0000","last_peered":"2024-09-21T15:36:05.218602+0000","last_clean":"2024-09-21T15:36:05.218602+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.218602+0000","last_undegraded":"2024-09-21T15:36:05.218602+0000","last_fullsized":"2024-09-21T15:36:05.218602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2407,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551938+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:04.551938+0000","last_peered":"2024-09-21T15:36:04.551938+0000","last_clean":"2024-09-21T15:36:04.551938+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:04.551938+0000","last_undegraded":"2024-09-21T15:36:04.551938+0000","last_fullsized":"2024-09-21T15:36:04.551938+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:33.057808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5123,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551793+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:04.551793+0000","last_peered":"2024-09-21T15:36:04.551793+0000","last_clean":"2024-09-21T15:36:04.551793+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:04.551793+0000","last_undegraded":"2024-09-21T15:36:04.551793+0000","last_fullsized":"2024-09-21T15:36:04.551793+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:13.889793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218741+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.218741+0000","last_peered":"2024-09-21T15:36:05.218741+0000","last_clean":"2024-09-21T15:36:05.218741+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.218741+0000","last_undegraded":"2024-09-21T15:36:05.218741+0000","last_fullsized":"2024-09-21T15:36:05.218741+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:28.468874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1101,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218711+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.218711+0000","last_peered":"2024-09-21T15:36:05.218711+0000","last_clean":"2024-09-21T15:36:05.218711+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.218711+0000","last_undegraded":"2024-09-21T15:36:05.218711+0000","last_fullsized":"2024-09-21T15:36:05.218711+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:32.430766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218761+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.218761+0000","last_peered":"2024-09-21T15:36:05.218761+0000","last_clean":"2024-09-21T15:36:05.218761+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.218761+0000","last_undegraded":"2024-09-21T15:36:05.218761+0000","last_fullsized":"2024-09-21T15:36:05.218761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:13.367520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2993,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218703+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.218703+0000","last_peered":"2024-09-21T15:36:05.218703+0000","last_clean":"2024-09-21T15:36:05.218703+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.218703+0000","last_undegraded":"2024-09-21T15:36:05.218703+0000","last_fullsized":"2024-09-21T15:36:05.218703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:34.219898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551739+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:04.551739+0000","last_peered":"2024-09-21T15:36:04.551739+0000","last_clean":"2024-09-21T15:36:04.551739+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:04.551739+0000","last_undegraded":"2024-09-21T15:36:04.551739+0000","last_fullsized":"2024-09-21T15:36:04.551739+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:30:54.191321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7848,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551805+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:04.551805+0000","last_peered":"2024-09-21T15:36:04.551805+0000","last_clean":"2024-09-21T15:36:04.551805+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:04.551805+0000","last_undegraded":"2024-09-21T15:36:04.551805+0000","last_fullsized":"2024-09-21T15:36:04.551805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:46.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.79300000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":1.022},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.96699999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.627},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.131}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.286},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.365}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.82199999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.876}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.036}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.056}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71599999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.2070000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.157},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.2330000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.075},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:09.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:10.111 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:10.128 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T15:36:09.694409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.526897"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5569,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551934+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:04.551934+0000","last_peered":"2024-09-21T15:36:04.551934+0000","last_clean":"2024-09-21T15:36:04.551934+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:04.551934+0000","last_undegraded":"2024-09-21T15:36:04.551934+0000","last_fullsized":"2024-09-21T15:36:04.551934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:12.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10066,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551953+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:04.551953+0000","last_peered":"2024-09-21T15:36:04.551953+0000","last_clean":"2024-09-21T15:36:04.551953+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:04.551953+0000","last_undegraded":"2024-09-21T15:36:04.551953+0000","last_fullsized":"2024-09-21T15:36:04.551953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:06.582849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33607,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218602+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.218602+0000","last_peered":"2024-09-21T15:36:05.218602+0000","last_clean":"2024-09-21T15:36:05.218602+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.218602+0000","last_undegraded":"2024-09-21T15:36:05.218602+0000","last_fullsized":"2024-09-21T15:36:05.218602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2407,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551938+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:04.551938+0000","last_peered":"2024-09-21T15:36:04.551938+0000","last_clean":"2024-09-21T15:36:04.551938+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:04.551938+0000","last_undegraded":"2024-09-21T15:36:04.551938+0000","last_fullsized":"2024-09-21T15:36:04.551938+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:33.057808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5123,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551793+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:04.551793+0000","last_peered":"2024-09-21T15:36:04.551793+0000","last_clean":"2024-09-21T15:36:04.551793+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:04.551793+0000","last_undegraded":"2024-09-21T15:36:04.551793+0000","last_fullsized":"2024-09-21T15:36:04.551793+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:13.889793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218741+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.218741+0000","last_peered":"2024-09-21T15:36:05.218741+0000","last_clean":"2024-09-21T15:36:05.218741+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.218741+0000","last_undegraded":"2024-09-21T15:36:05.218741+0000","last_fullsized":"2024-09-21T15:36:05.218741+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:28.468874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1101,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218711+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.218711+0000","last_peered":"2024-09-21T15:36:05.218711+0000","last_clean":"2024-09-21T15:36:05.218711+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.218711+0000","last_undegraded":"2024-09-21T15:36:05.218711+0000","last_fullsized":"2024-09-21T15:36:05.218711+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:32.430766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218761+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.218761+0000","last_peered":"2024-09-21T15:36:05.218761+0000","last_clean":"2024-09-21T15:36:05.218761+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.218761+0000","last_undegraded":"2024-09-21T15:36:05.218761+0000","last_fullsized":"2024-09-21T15:36:05.218761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:13.367520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2993,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218703+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.218703+0000","last_peered":"2024-09-21T15:36:05.218703+0000","last_clean":"2024-09-21T15:36:05.218703+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.218703+0000","last_undegraded":"2024-09-21T15:36:05.218703+0000","last_fullsized":"2024-09-21T15:36:05.218703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:34.219898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551739+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:04.551739+0000","last_peered":"2024-09-21T15:36:04.551739+0000","last_clean":"2024-09-21T15:36:04.551739+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:04.551739+0000","last_undegraded":"2024-09-21T15:36:04.551739+0000","last_fullsized":"2024-09-21T15:36:04.551739+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:30:54.191321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7848,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551805+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:04.551805+0000","last_peered":"2024-09-21T15:36:04.551805+0000","last_clean":"2024-09-21T15:36:04.551805+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:04.551805+0000","last_undegraded":"2024-09-21T15:36:04.551805+0000","last_fullsized":"2024-09-21T15:36:04.551805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:46.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.79300000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":1.022},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.96699999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.627},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.131}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.286},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.365}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.82199999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.876}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.036}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.056}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71599999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.2070000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.157},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.2330000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.075},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:10.130 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T15:36:10.131 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T15:36:10.426 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T15:36:10.436 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T15:36:10.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:10.713 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:10.730 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T15:36:09.694409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.526897"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5569,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551934+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:04.551934+0000","last_peered":"2024-09-21T15:36:04.551934+0000","last_clean":"2024-09-21T15:36:04.551934+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:04.551934+0000","last_undegraded":"2024-09-21T15:36:04.551934+0000","last_fullsized":"2024-09-21T15:36:04.551934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:12.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10066,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551953+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:04.551953+0000","last_peered":"2024-09-21T15:36:04.551953+0000","last_clean":"2024-09-21T15:36:04.551953+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:04.551953+0000","last_undegraded":"2024-09-21T15:36:04.551953+0000","last_fullsized":"2024-09-21T15:36:04.551953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:06.582849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33607,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218602+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.218602+0000","last_peered":"2024-09-21T15:36:05.218602+0000","last_clean":"2024-09-21T15:36:05.218602+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.218602+0000","last_undegraded":"2024-09-21T15:36:05.218602+0000","last_fullsized":"2024-09-21T15:36:05.218602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2407,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551938+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:04.551938+0000","last_peered":"2024-09-21T15:36:04.551938+0000","last_clean":"2024-09-21T15:36:04.551938+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:04.551938+0000","last_undegraded":"2024-09-21T15:36:04.551938+0000","last_fullsized":"2024-09-21T15:36:04.551938+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:33.057808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5123,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551793+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:04.551793+0000","last_peered":"2024-09-21T15:36:04.551793+0000","last_clean":"2024-09-21T15:36:04.551793+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:04.551793+0000","last_undegraded":"2024-09-21T15:36:04.551793+0000","last_fullsized":"2024-09-21T15:36:04.551793+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:13.889793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218741+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.218741+0000","last_peered":"2024-09-21T15:36:05.218741+0000","last_clean":"2024-09-21T15:36:05.218741+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.218741+0000","last_undegraded":"2024-09-21T15:36:05.218741+0000","last_fullsized":"2024-09-21T15:36:05.218741+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:28.468874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1101,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218711+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.218711+0000","last_peered":"2024-09-21T15:36:05.218711+0000","last_clean":"2024-09-21T15:36:05.218711+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.218711+0000","last_undegraded":"2024-09-21T15:36:05.218711+0000","last_fullsized":"2024-09-21T15:36:05.218711+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:32.430766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218761+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.218761+0000","last_peered":"2024-09-21T15:36:05.218761+0000","last_clean":"2024-09-21T15:36:05.218761+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.218761+0000","last_undegraded":"2024-09-21T15:36:05.218761+0000","last_fullsized":"2024-09-21T15:36:05.218761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:13.367520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2993,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218703+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.218703+0000","last_peered":"2024-09-21T15:36:05.218703+0000","last_clean":"2024-09-21T15:36:05.218703+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.218703+0000","last_undegraded":"2024-09-21T15:36:05.218703+0000","last_fullsized":"2024-09-21T15:36:05.218703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:34.219898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551739+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:04.551739+0000","last_peered":"2024-09-21T15:36:04.551739+0000","last_clean":"2024-09-21T15:36:04.551739+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:04.551739+0000","last_undegraded":"2024-09-21T15:36:04.551739+0000","last_fullsized":"2024-09-21T15:36:04.551739+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:30:54.191321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7848,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551805+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:04.551805+0000","last_peered":"2024-09-21T15:36:04.551805+0000","last_clean":"2024-09-21T15:36:04.551805+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:04.551805+0000","last_undegraded":"2024-09-21T15:36:04.551805+0000","last_fullsized":"2024-09-21T15:36:04.551805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:46.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.79300000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":1.022},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.96699999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.627},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.131}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.286},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.365}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.82199999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.876}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.036}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.056}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71599999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.2070000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.157},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.2330000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.075},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:10.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:11.010 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:11.010 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:11.026 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T15:36:09.694409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.526897"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5569,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551934+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:04.551934+0000","last_peered":"2024-09-21T15:36:04.551934+0000","last_clean":"2024-09-21T15:36:04.551934+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:04.551934+0000","last_undegraded":"2024-09-21T15:36:04.551934+0000","last_fullsized":"2024-09-21T15:36:04.551934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:12.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10066,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551953+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:04.551953+0000","last_peered":"2024-09-21T15:36:04.551953+0000","last_clean":"2024-09-21T15:36:04.551953+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:04.551953+0000","last_undegraded":"2024-09-21T15:36:04.551953+0000","last_fullsized":"2024-09-21T15:36:04.551953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:06.582849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33607,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218602+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.218602+0000","last_peered":"2024-09-21T15:36:05.218602+0000","last_clean":"2024-09-21T15:36:05.218602+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.218602+0000","last_undegraded":"2024-09-21T15:36:05.218602+0000","last_fullsized":"2024-09-21T15:36:05.218602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2407,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551938+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:04.551938+0000","last_peered":"2024-09-21T15:36:04.551938+0000","last_clean":"2024-09-21T15:36:04.551938+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:04.551938+0000","last_undegraded":"2024-09-21T15:36:04.551938+0000","last_fullsized":"2024-09-21T15:36:04.551938+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:33.057808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5123,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551793+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:04.551793+0000","last_peered":"2024-09-21T15:36:04.551793+0000","last_clean":"2024-09-21T15:36:04.551793+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:04.551793+0000","last_undegraded":"2024-09-21T15:36:04.551793+0000","last_fullsized":"2024-09-21T15:36:04.551793+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:13.889793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218741+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.218741+0000","last_peered":"2024-09-21T15:36:05.218741+0000","last_clean":"2024-09-21T15:36:05.218741+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.218741+0000","last_undegraded":"2024-09-21T15:36:05.218741+0000","last_fullsized":"2024-09-21T15:36:05.218741+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:28.468874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1101,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218711+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.218711+0000","last_peered":"2024-09-21T15:36:05.218711+0000","last_clean":"2024-09-21T15:36:05.218711+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.218711+0000","last_undegraded":"2024-09-21T15:36:05.218711+0000","last_fullsized":"2024-09-21T15:36:05.218711+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:32.430766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218761+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.218761+0000","last_peered":"2024-09-21T15:36:05.218761+0000","last_clean":"2024-09-21T15:36:05.218761+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.218761+0000","last_undegraded":"2024-09-21T15:36:05.218761+0000","last_fullsized":"2024-09-21T15:36:05.218761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:13.367520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2993,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218703+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.218703+0000","last_peered":"2024-09-21T15:36:05.218703+0000","last_clean":"2024-09-21T15:36:05.218703+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.218703+0000","last_undegraded":"2024-09-21T15:36:05.218703+0000","last_fullsized":"2024-09-21T15:36:05.218703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:34.219898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551739+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:04.551739+0000","last_peered":"2024-09-21T15:36:04.551739+0000","last_clean":"2024-09-21T15:36:04.551739+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:04.551739+0000","last_undegraded":"2024-09-21T15:36:04.551739+0000","last_fullsized":"2024-09-21T15:36:04.551739+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:30:54.191321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7848,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551805+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:04.551805+0000","last_peered":"2024-09-21T15:36:04.551805+0000","last_clean":"2024-09-21T15:36:04.551805+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:04.551805+0000","last_undegraded":"2024-09-21T15:36:04.551805+0000","last_fullsized":"2024-09-21T15:36:04.551805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:46.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.79300000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":1.022},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.96699999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.627},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.131}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.286},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.365}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.82199999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.876}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.036}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.056}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71599999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.2070000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.157},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.2330000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.075},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:11.029 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T15:36:11.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:11.324 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:11.325 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:11.342 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-21T15:36:09.694409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.526897"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5569,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551934+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:04.551934+0000","last_peered":"2024-09-21T15:36:04.551934+0000","last_clean":"2024-09-21T15:36:04.551934+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:04.551934+0000","last_undegraded":"2024-09-21T15:36:04.551934+0000","last_fullsized":"2024-09-21T15:36:04.551934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:12.538199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10066,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551953+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:04.551953+0000","last_peered":"2024-09-21T15:36:04.551953+0000","last_clean":"2024-09-21T15:36:04.551953+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:04.551953+0000","last_undegraded":"2024-09-21T15:36:04.551953+0000","last_fullsized":"2024-09-21T15:36:04.551953+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:06.582849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33607,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218602+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.218602+0000","last_peered":"2024-09-21T15:36:05.218602+0000","last_clean":"2024-09-21T15:36:05.218602+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.218602+0000","last_undegraded":"2024-09-21T15:36:05.218602+0000","last_fullsized":"2024-09-21T15:36:05.218602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:34:37.113724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2407,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551938+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:04.551938+0000","last_peered":"2024-09-21T15:36:04.551938+0000","last_clean":"2024-09-21T15:36:04.551938+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:04.551938+0000","last_undegraded":"2024-09-21T15:36:04.551938+0000","last_fullsized":"2024-09-21T15:36:04.551938+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:33.057808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5123,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551793+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:04.551793+0000","last_peered":"2024-09-21T15:36:04.551793+0000","last_clean":"2024-09-21T15:36:04.551793+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:04.551793+0000","last_undegraded":"2024-09-21T15:36:04.551793+0000","last_fullsized":"2024-09-21T15:36:04.551793+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:20:13.889793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218741+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.218741+0000","last_peered":"2024-09-21T15:36:05.218741+0000","last_clean":"2024-09-21T15:36:05.218741+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.218741+0000","last_undegraded":"2024-09-21T15:36:05.218741+0000","last_fullsized":"2024-09-21T15:36:05.218741+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:42:28.468874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1101,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218711+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.218711+0000","last_peered":"2024-09-21T15:36:05.218711+0000","last_clean":"2024-09-21T15:36:05.218711+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.218711+0000","last_undegraded":"2024-09-21T15:36:05.218711+0000","last_fullsized":"2024-09-21T15:36:05.218711+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:32.430766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218761+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.218761+0000","last_peered":"2024-09-21T15:36:05.218761+0000","last_clean":"2024-09-21T15:36:05.218761+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.218761+0000","last_undegraded":"2024-09-21T15:36:05.218761+0000","last_fullsized":"2024-09-21T15:36:05.218761+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:13.367520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2993,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.218703+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.218703+0000","last_peered":"2024-09-21T15:36:05.218703+0000","last_clean":"2024-09-21T15:36:05.218703+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.218703+0000","last_undegraded":"2024-09-21T15:36:05.218703+0000","last_fullsized":"2024-09-21T15:36:05.218703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:34.219898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3168,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551739+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:04.551739+0000","last_peered":"2024-09-21T15:36:04.551739+0000","last_clean":"2024-09-21T15:36:04.551739+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:04.551739+0000","last_undegraded":"2024-09-21T15:36:04.551739+0000","last_fullsized":"2024-09-21T15:36:04.551739+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:30:54.191321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7848,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.551805+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:04.551805+0000","last_peered":"2024-09-21T15:36:04.551805+0000","last_clean":"2024-09-21T15:36:04.551805+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:04.551805+0000","last_undegraded":"2024-09-21T15:36:04.551805+0000","last_fullsized":"2024-09-21T15:36:04.551805+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:42:46.876207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.79300000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":1.022},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.96699999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.627},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":1.131}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.286},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.365}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.82199999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.94799999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.78200000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.876}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.036}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.056}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71599999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1599999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2609999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.2070000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":1.157},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.2330000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.995}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.316}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.37},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.64300000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.76300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.075},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.78000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.1020000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701960,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:11.344 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T15:36:11.618 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:11.619 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":181,"fsid":"09582c7a-ec34-4064-810b-b65e59e21b91","created":"2024-09-21T15:32:16.991419+0000","modified":"2024-09-21T15:36:05.186948+0000","last_up_change":"2024-09-21T15:35:44.012702+0000","last_in_change":"2024-09-21T15:35:49.035022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T15:32:29.715035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T15:32:35.528036+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T15:32:44.899492+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":160,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2f9d147-bbb0-470e-8441-d891316be3ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":143,"up_from":162,"up_thru":171,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6825","nonce":997327856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6827","nonce":997327856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6831","nonce":997327856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":997327856},{"type":"v1","addr":"172.21.15.107:6829","nonce":997327856}]},"public_addr":"172.21.15.107:6825/997327856","cluster_addr":"172.21.15.107:6827/997327856","heartbeat_back_addr":"172.21.15.107:6831/997327856","heartbeat_front_addr":"172.21.15.107:6829/997327856","state":["exists","up"]},{"osd":1,"uuid":"e13f399d-5ddd-47e1-879e-eda2f7ec930f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6817","nonce":615179745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6819","nonce":615179745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6823","nonce":615179745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":615179745},{"type":"v1","addr":"172.21.15.107:6821","nonce":615179745}]},"public_addr":"172.21.15.107:6817/615179745","cluster_addr":"172.21.15.107:6819/615179745","heartbeat_back_addr":"172.21.15.107:6823/615179745","heartbeat_front_addr":"172.21.15.107:6821/615179745","state":["exists","up"]},{"osd":2,"uuid":"84d52d47-67ab-43ca-a983-5568d9cf1975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6801","nonce":3712995483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6803","nonce":3712995483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6807","nonce":3712995483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3712995483},{"type":"v1","addr":"172.21.15.107:6805","nonce":3712995483}]},"public_addr":"172.21.15.107:6801/3712995483","cluster_addr":"172.21.15.107:6803/3712995483","heartbeat_back_addr":"172.21.15.107:6807/3712995483","heartbeat_front_addr":"172.21.15.107:6805/3712995483","state":["exists","up"]},{"osd":3,"uuid":"adbb6235-d8c3-4b20-a7e4-0046d22ed5fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6809","nonce":3212640731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6811","nonce":3212640731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6815","nonce":3212640731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3212640731},{"type":"v1","addr":"172.21.15.107:6813","nonce":3212640731}]},"public_addr":"172.21.15.107:6809/3212640731","cluster_addr":"172.21.15.107:6811/3212640731","heartbeat_back_addr":"172.21.15.107:6815/3212640731","heartbeat_front_addr":"172.21.15.107:6813/3212640731","state":["exists","up"]},{"osd":4,"uuid":"f3780990-ed73-4f96-b93e-34e83e63d9e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6801","nonce":1739937222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6803","nonce":1739937222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6807","nonce":1739937222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1739937222},{"type":"v1","addr":"172.21.15.170:6805","nonce":1739937222}]},"public_addr":"172.21.15.170:6801/1739937222","cluster_addr":"172.21.15.170:6803/1739937222","heartbeat_back_addr":"172.21.15.170:6807/1739937222","heartbeat_front_addr":"172.21.15.170:6805/1739937222","state":["exists","up"]},{"osd":5,"uuid":"36f2dd75-f924-443d-ac44-02c8d0fa6cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6824","nonce":4269648595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6825","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6827","nonce":4269648595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6831","nonce":4269648595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4269648595},{"type":"v1","addr":"172.21.15.170:6829","nonce":4269648595}]},"public_addr":"172.21.15.170:6824/4269648595","cluster_addr":"172.21.15.170:6827/4269648595","heartbeat_back_addr":"172.21.15.170:6831/4269648595","heartbeat_front_addr":"172.21.15.170:6829/4269648595","state":["exists","up"]},{"osd":6,"uuid":"cae6719a-cf33-472a-b127-964ed06f3c5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6810","nonce":3971003516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6813","nonce":3971003516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6819","nonce":3971003516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3971003516},{"type":"v1","addr":"172.21.15.170:6816","nonce":3971003516}]},"public_addr":"172.21.15.170:6810/3971003516","cluster_addr":"172.21.15.170:6813/3971003516","heartbeat_back_addr":"172.21.15.170:6819/3971003516","heartbeat_front_addr":"172.21.15.170:6816/3971003516","state":["exists","up"]},{"osd":7,"uuid":"dde8b30b-cb83-4b0e-bdd9-b54ae839d666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6811","nonce":3076528025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6818","nonce":3076528025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6826","nonce":3076528025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3076528025},{"type":"v1","addr":"172.21.15.170:6821","nonce":3076528025}]},"public_addr":"172.21.15.170:6811/3076528025","cluster_addr":"172.21.15.170:6818/3076528025","heartbeat_back_addr":"172.21.15.170:6826/3076528025","heartbeat_front_addr":"172.21.15.170:6821/3076528025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.112615+0000","dead_epoch":144},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.099474+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.100534+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.993492+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:24.901492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.197653+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.192998+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T15:32:25.045812+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.b","osds":[7,6]},{"pgid":"3.14","osds":[2,6]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T15:36:12.637 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T15:36:12.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:12.796 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:12.796 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-21T15:36:12.796 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:12.812 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T15:36:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T15:36:13.115 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T15:36:13.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:13.262 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:13.263 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:13.263 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:13.278 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T15:36:13.556 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T15:36:13.574 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T15:36:13.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:13.713 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:13.713 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:13.713 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:13.728 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T15:36:13.997 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T15:36:14.016 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T15:36:14.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:14.149 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:14.149 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:14.149 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:14.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T15:36:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T15:36:14.466 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T15:36:14.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:14.618 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:14.618 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:14.618 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:14.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T15:36:14.905 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T15:36:14.923 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T15:36:14.923 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:15.073 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:15.073 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:15.073 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:15.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T15:36:15.370 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T15:36:15.388 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T15:36:15.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:15.542 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:15.543 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:15.543 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:15.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T15:36:15.848 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T15:36:15.867 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T15:36:15.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T15:36:16.007 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T15:36:16.008 INFO:teuthology.orchestra.run.smithi107.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T15:36:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T15:36:16.022 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T15:36:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T15:36:16.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:16.605 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:16.606 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:16.623 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-21T15:36:15.696111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15473,"ondisk_log_size":15473,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340864,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":322469,"kb_avail":754633856,"statfs":{"total":773094113280,"available":772745068544,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":330209154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.505665"},"pg_stats":[{"pgid":"3.1f","version":"160'666","reported_seq":5570,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519253+0000","last_change":"2024-09-21T15:35:53.783985+0000","last_active":"2024-09-21T15:36:06.519253+0000","last_peered":"2024-09-21T15:36:06.519253+0000","last_clean":"2024-09-21T15:36:06.519253+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:06.519253+0000","last_undegraded":"2024-09-21T15:36:06.519253+0000","last_fullsized":"2024-09-21T15:36:06.519253+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"154'505","reported_seq":7387,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128066+0000","last_change":"2024-09-21T15:35:55.791854+0000","last_active":"2024-09-21T15:36:04.128066+0000","last_peered":"2024-09-21T15:36:04.128066+0000","last_clean":"2024-09-21T15:36:04.128066+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:04.128066+0000","last_undegraded":"2024-09-21T15:36:04.128066+0000","last_fullsized":"2024-09-21T15:36:04.128066+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:25.301570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":734,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200729+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.200729+0000","last_peered":"2024-09-21T15:36:05.200729+0000","last_clean":"2024-09-21T15:36:05.200729+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.200729+0000","last_undegraded":"2024-09-21T15:36:05.200729+0000","last_fullsized":"2024-09-21T15:36:05.200729+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:48.281855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6098,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200382+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.200382+0000","last_peered":"2024-09-21T15:36:05.200382+0000","last_clean":"2024-09-21T15:36:05.200382+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.200382+0000","last_undegraded":"2024-09-21T15:36:05.200382+0000","last_fullsized":"2024-09-21T15:36:05.200382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:12.470168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"159'427","reported_seq":10067,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519345+0000","last_change":"2024-09-21T15:35:52.777459+0000","last_active":"2024-09-21T15:36:06.519345+0000","last_peered":"2024-09-21T15:36:06.519345+0000","last_clean":"2024-09-21T15:36:06.519345+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:06.519345+0000","last_undegraded":"2024-09-21T15:36:06.519345+0000","last_fullsized":"2024-09-21T15:36:06.519345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'262","reported_seq":7368,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190694+0000","last_change":"2024-09-21T15:35:41.761171+0000","last_active":"2024-09-21T15:36:04.190694+0000","last_peered":"2024-09-21T15:36:04.190694+0000","last_clean":"2024-09-21T15:36:04.190694+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:04.190694+0000","last_undegraded":"2024-09-21T15:36:04.190694+0000","last_fullsized":"2024-09-21T15:36:04.190694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:58.465471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'637","reported_seq":33608,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.219411+0000","last_change":"2024-09-21T15:35:41.900749+0000","last_active":"2024-09-21T15:36:05.219411+0000","last_peered":"2024-09-21T15:36:05.219411+0000","last_clean":"2024-09-21T15:36:05.219411+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:05.219411+0000","last_undegraded":"2024-09-21T15:36:05.219411+0000","last_fullsized":"2024-09-21T15:36:05.219411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'490","reported_seq":1364,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127674+0000","last_change":"2024-09-21T15:35:55.784539+0000","last_active":"2024-09-21T15:36:04.127674+0000","last_peered":"2024-09-21T15:36:04.127674+0000","last_clean":"2024-09-21T15:36:04.127674+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:04.127674+0000","last_undegraded":"2024-09-21T15:36:04.127674+0000","last_fullsized":"2024-09-21T15:36:04.127674+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:32:42.949589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"159'688","reported_seq":2408,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519447+0000","last_change":"2024-09-21T15:35:55.786567+0000","last_active":"2024-09-21T15:36:06.519447+0000","last_peered":"2024-09-21T15:36:06.519447+0000","last_clean":"2024-09-21T15:36:06.519447+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:06.519447+0000","last_undegraded":"2024-09-21T15:36:06.519447+0000","last_fullsized":"2024-09-21T15:36:06.519447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"156'368","reported_seq":6167,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200498+0000","last_change":"2024-09-21T15:35:41.780758+0000","last_active":"2024-09-21T15:36:05.200498+0000","last_peered":"2024-09-21T15:36:05.200498+0000","last_clean":"2024-09-21T15:36:05.200498+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:05.200498+0000","last_undegraded":"2024-09-21T15:36:05.200498+0000","last_fullsized":"2024-09-21T15:36:05.200498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:35.036579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'309","reported_seq":5317,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303251+0000","last_change":"2024-09-21T15:35:41.903955+0000","last_active":"2024-09-21T15:36:05.303251+0000","last_peered":"2024-09-21T15:36:05.303251+0000","last_clean":"2024-09-21T15:36:05.303251+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:05.303251+0000","last_undegraded":"2024-09-21T15:36:05.303251+0000","last_fullsized":"2024-09-21T15:36:05.303251+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:21:54.055236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"159'726","reported_seq":4705,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193000+0000","last_change":"2024-09-21T15:35:41.764683+0000","last_active":"2024-09-21T15:36:04.193000+0000","last_peered":"2024-09-21T15:36:04.193000+0000","last_clean":"2024-09-21T15:36:04.193000+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:04.193000+0000","last_undegraded":"2024-09-21T15:36:04.193000+0000","last_fullsized":"2024-09-21T15:36:04.193000+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:35:24.898253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799532+0000","last_change":"2024-09-21T15:34:30.770301+0000","last_active":"2024-09-21T15:36:05.799532+0000","last_peered":"2024-09-21T15:36:05.799532+0000","last_clean":"2024-09-21T15:36:05.799532+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:05.799532+0000","last_undegraded":"2024-09-21T15:36:05.799532+0000","last_fullsized":"2024-09-21T15:36:05.799532+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:39.753057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'579","reported_seq":5124,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519286+0000","last_change":"2024-09-21T15:35:52.776491+0000","last_active":"2024-09-21T15:36:06.519286+0000","last_peered":"2024-09-21T15:36:06.519286+0000","last_clean":"2024-09-21T15:36:06.519286+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:06.519286+0000","last_undegraded":"2024-09-21T15:36:06.519286+0000","last_fullsized":"2024-09-21T15:36:06.519286+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.219476+0000","last_change":"2024-09-21T15:35:52.753036+0000","last_active":"2024-09-21T15:36:05.219476+0000","last_peered":"2024-09-21T15:36:05.219476+0000","last_clean":"2024-09-21T15:36:05.219476+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:05.219476+0000","last_undegraded":"2024-09-21T15:36:05.219476+0000","last_fullsized":"2024-09-21T15:36:05.219476+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"159'325","reported_seq":1682,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.128050+0000","last_change":"2024-09-21T15:35:55.789871+0000","last_active":"2024-09-21T15:36:04.128050+0000","last_peered":"2024-09-21T15:36:04.128050+0000","last_clean":"2024-09-21T15:36:04.128050+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:04.128050+0000","last_undegraded":"2024-09-21T15:36:04.128050+0000","last_fullsized":"2024-09-21T15:36:04.128050+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:17:53.214044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303496+0000","last_change":"2024-09-21T15:35:50.059942+0000","last_active":"2024-09-21T15:36:05.303496+0000","last_peered":"2024-09-21T15:36:05.303496+0000","last_clean":"2024-09-21T15:36:05.303496+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:05.303496+0000","last_undegraded":"2024-09-21T15:36:05.303496+0000","last_fullsized":"2024-09-21T15:36:05.303496+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:53.098492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"160'232","reported_seq":1102,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.219380+0000","last_change":"2024-09-21T15:35:51.776984+0000","last_active":"2024-09-21T15:36:05.219380+0000","last_peered":"2024-09-21T15:36:05.219380+0000","last_clean":"2024-09-21T15:36:05.219380+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:05.219380+0000","last_undegraded":"2024-09-21T15:36:05.219380+0000","last_fullsized":"2024-09-21T15:36:05.219380+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.219379+0000","last_change":"2024-09-21T15:35:51.778336+0000","last_active":"2024-09-21T15:36:05.219379+0000","last_peered":"2024-09-21T15:36:05.219379+0000","last_clean":"2024-09-21T15:36:05.219379+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:05.219379+0000","last_undegraded":"2024-09-21T15:36:05.219379+0000","last_fullsized":"2024-09-21T15:36:05.219379+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"159'783","reported_seq":1554,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200872+0000","last_change":"2024-09-21T15:35:41.779595+0000","last_active":"2024-09-21T15:36:05.200872+0000","last_peered":"2024-09-21T15:36:05.200872+0000","last_clean":"2024-09-21T15:36:05.200872+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:05.200872+0000","last_undegraded":"2024-09-21T15:36:05.200872+0000","last_fullsized":"2024-09-21T15:36:05.200872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:05:50.057113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"160'450","reported_seq":7091,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190858+0000","last_change":"2024-09-21T15:35:55.787876+0000","last_active":"2024-09-21T15:36:04.190858+0000","last_peered":"2024-09-21T15:36:04.190858+0000","last_clean":"2024-09-21T15:36:04.190858+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:04.190858+0000","last_undegraded":"2024-09-21T15:36:04.190858+0000","last_fullsized":"2024-09-21T15:36:04.190858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:43.158045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":178,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200533+0000","last_change":"2024-09-21T15:32:38.802783+0000","last_active":"2024-09-21T15:36:05.200533+0000","last_peered":"2024-09-21T15:36:05.200533+0000","last_clean":"2024-09-21T15:36:05.200533+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:05.200533+0000","last_undegraded":"2024-09-21T15:36:05.200533+0000","last_fullsized":"2024-09-21T15:36:05.200533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.868038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"159'935","reported_seq":2994,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.219318+0000","last_change":"2024-09-21T15:35:51.777323+0000","last_active":"2024-09-21T15:36:05.219318+0000","last_peered":"2024-09-21T15:36:05.219318+0000","last_clean":"2024-09-21T15:36:05.219318+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:05.219318+0000","last_undegraded":"2024-09-21T15:36:05.219318+0000","last_fullsized":"2024-09-21T15:36:05.219318+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.193020+0000","last_change":"2024-09-21T15:33:21.066980+0000","last_active":"2024-09-21T15:36:04.193020+0000","last_peered":"2024-09-21T15:36:04.193020+0000","last_clean":"2024-09-21T15:36:04.193020+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:04.193020+0000","last_undegraded":"2024-09-21T15:36:04.193020+0000","last_fullsized":"2024-09-21T15:36:04.193020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:56:00.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'533","reported_seq":3169,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519216+0000","last_change":"2024-09-21T15:35:52.778866+0000","last_active":"2024-09-21T15:36:06.519216+0000","last_peered":"2024-09-21T15:36:06.519216+0000","last_clean":"2024-09-21T15:36:06.519216+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:06.519216+0000","last_undegraded":"2024-09-21T15:36:06.519216+0000","last_fullsized":"2024-09-21T15:36:06.519216+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303160+0000","last_change":"2024-09-21T15:32:39.777106+0000","last_active":"2024-09-21T15:36:05.303160+0000","last_peered":"2024-09-21T15:36:05.303160+0000","last_clean":"2024-09-21T15:36:05.303160+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:05.303160+0000","last_undegraded":"2024-09-21T15:36:05.303160+0000","last_fullsized":"2024-09-21T15:36:05.303160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:33:22.280623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'405","reported_seq":1262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127972+0000","last_change":"2024-09-21T15:35:55.785142+0000","last_active":"2024-09-21T15:36:04.127972+0000","last_peered":"2024-09-21T15:36:04.127972+0000","last_clean":"2024-09-21T15:36:04.127972+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:04.127972+0000","last_undegraded":"2024-09-21T15:36:04.127972+0000","last_fullsized":"2024-09-21T15:36:04.127972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:03.102157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":201,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200390+0000","last_change":"2024-09-21T15:33:21.071963+0000","last_active":"2024-09-21T15:36:05.200390+0000","last_peered":"2024-09-21T15:36:05.200390+0000","last_clean":"2024-09-21T15:36:05.200390+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:05.200390+0000","last_undegraded":"2024-09-21T15:36:05.200390+0000","last_fullsized":"2024-09-21T15:36:05.200390+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:35.753712+0000","last_clean_scrub_stamp":"2024-09-21T15:32:35.753712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:29.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":353,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303232+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:05.303232+0000","last_peered":"2024-09-21T15:36:05.303232+0000","last_clean":"2024-09-21T15:36:05.303232+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:05.303232+0000","last_undegraded":"2024-09-21T15:36:05.303232+0000","last_fullsized":"2024-09-21T15:36:05.303232+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:00.100466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"156'611","reported_seq":1891,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.127835+0000","last_change":"2024-09-21T15:35:55.786321+0000","last_active":"2024-09-21T15:36:04.127835+0000","last_peered":"2024-09-21T15:36:04.127835+0000","last_clean":"2024-09-21T15:36:04.127835+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:04.127835+0000","last_undegraded":"2024-09-21T15:36:04.127835+0000","last_fullsized":"2024-09-21T15:36:04.127835+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:07:43.865453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'551","reported_seq":1954,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:04.190759+0000","last_change":"2024-09-21T15:35:41.762448+0000","last_active":"2024-09-21T15:36:04.190759+0000","last_peered":"2024-09-21T15:36:04.190759+0000","last_clean":"2024-09-21T15:36:04.190759+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:04.190759+0000","last_undegraded":"2024-09-21T15:36:04.190759+0000","last_fullsized":"2024-09-21T15:36:04.190759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:01:29.960487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'384","reported_seq":5192,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799354+0000","last_change":"2024-09-21T15:35:41.768974+0000","last_active":"2024-09-21T15:36:05.799354+0000","last_peered":"2024-09-21T15:36:05.799354+0000","last_clean":"2024-09-21T15:36:05.799354+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:05.799354+0000","last_undegraded":"2024-09-21T15:36:05.799354+0000","last_fullsized":"2024-09-21T15:36:05.799354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:40:19.042233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9490,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303549+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.303549+0000","last_peered":"2024-09-21T15:36:05.303549+0000","last_clean":"2024-09-21T15:36:05.303549+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.303549+0000","last_undegraded":"2024-09-21T15:36:05.303549+0000","last_fullsized":"2024-09-21T15:36:05.303549+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:29.221657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"160'312","reported_seq":2310,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.200619+0000","last_change":"2024-09-21T15:35:41.778709+0000","last_active":"2024-09-21T15:36:05.200619+0000","last_peered":"2024-09-21T15:36:05.200619+0000","last_clean":"2024-09-21T15:36:05.200619+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:05.200619+0000","last_undegraded":"2024-09-21T15:36:05.200619+0000","last_fullsized":"2024-09-21T15:36:05.200619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:07.679282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"158'395","reported_seq":17624,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303356+0000","last_change":"2024-09-21T15:35:41.905493+0000","last_active":"2024-09-21T15:36:05.303356+0000","last_peered":"2024-09-21T15:36:05.303356+0000","last_clean":"2024-09-21T15:36:05.303356+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:05.303356+0000","last_undegraded":"2024-09-21T15:36:05.303356+0000","last_fullsized":"2024-09-21T15:36:05.303356+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:15.634878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"159'376","reported_seq":38175,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303525+0000","last_change":"2024-09-21T15:35:41.904539+0000","last_active":"2024-09-21T15:36:05.303525+0000","last_peered":"2024-09-21T15:36:05.303525+0000","last_clean":"2024-09-21T15:36:05.303525+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:05.303525+0000","last_undegraded":"2024-09-21T15:36:05.303525+0000","last_fullsized":"2024-09-21T15:36:05.303525+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:15:09.154094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2089,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303647+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.303647+0000","last_peered":"2024-09-21T15:36:05.303647+0000","last_clean":"2024-09-21T15:36:05.303647+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.303647+0000","last_undegraded":"2024-09-21T15:36:05.303647+0000","last_fullsized":"2024-09-21T15:36:05.303647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:18.790976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'302","reported_seq":1506,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.799336+0000","last_change":"2024-09-21T15:35:54.786465+0000","last_active":"2024-09-21T15:36:05.799336+0000","last_peered":"2024-09-21T15:36:05.799336+0000","last_clean":"2024-09-21T15:36:05.799336+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:05.799336+0000","last_undegraded":"2024-09-21T15:36:05.799336+0000","last_fullsized":"2024-09-21T15:36:05.799336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:22:19.076196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"159'619","reported_seq":6837,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303132+0000","last_change":"2024-09-21T15:35:41.902820+0000","last_active":"2024-09-21T15:36:05.303132+0000","last_peered":"2024-09-21T15:36:05.303132+0000","last_clean":"2024-09-21T15:36:05.303132+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:05.303132+0000","last_undegraded":"2024-09-21T15:36:05.303132+0000","last_fullsized":"2024-09-21T15:36:05.303132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:15:01.202517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"159'612","reported_seq":7849,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:06.519214+0000","last_change":"2024-09-21T15:35:54.790682+0000","last_active":"2024-09-21T15:36:06.519214+0000","last_peered":"2024-09-21T15:36:06.519214+0000","last_clean":"2024-09-21T15:36:06.519214+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:06.519214+0000","last_undegraded":"2024-09-21T15:36:06.519214+0000","last_fullsized":"2024-09-21T15:36:06.519214+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"159'488","reported_seq":10990,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.303379+0000","last_change":"2024-09-21T15:35:41.905167+0000","last_active":"2024-09-21T15:36:05.303379+0000","last_peered":"2024-09-21T15:36:05.303379+0000","last_clean":"2024-09-21T15:36:05.303379+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:05.303379+0000","last_undegraded":"2024-09-21T15:36:05.303379+0000","last_fullsized":"2024-09-21T15:36:05.303379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:34.594808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15353,"ondisk_log_size":15353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.70899999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.89100000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.751},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81899999999999995}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":0.91300000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":0.97099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45544,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326296,"statfs":{"total":96636764160,"available":96590127104,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.82499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":0.66000000000000003}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.95299999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.97799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":1.234}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.27},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.1379999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55404,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316436,"statfs":{"total":96636764160,"available":96580030464,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.747},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.91700000000000004}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.84499999999999997}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.038},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.88}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.1000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":0.56399999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.70399999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.622},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.52600000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.878},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.81699999999999995}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.069}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":1.046},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0149999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.93799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55396,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316444,"statfs":{"total":96636764160,"available":96580038656,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.62},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.5}]},{"osd":1,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.27700000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.115,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.33000000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.0660000000000001,"5min":1.127,"15min":1.127},"last":0.47799999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.69299999999999995},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2050000000000001,"5min":1.802,"15min":1.802},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68700000000000006,"15min":0.72199999999999998},"min":{"1min":0.35699999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.874,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.54000000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.80300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93100000000000005,"5min":1.587,"15min":1.587},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.1559999999999999,"15min":1.022},"min":{"1min":0.50600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.23,"5min":24.064,"15min":24.064},"last":0.79700000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.85599999999999998,"15min":0.91600000000000004},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1160000000000001,"5min":1.966,"15min":1.966},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.91700000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.351,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.93100000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.407,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1379999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.95999999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.97999999999999998},"min":{"1min":0.65000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.286,"5min":1.772,"15min":1.772},"last":1.169},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.94799999999999995,"15min":1.01},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.3169999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.3300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.629}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.89800000000000002}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.79900000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.87},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0860000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":1.163},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":0.83799999999999997}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":0.96999999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":1.3}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29168,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342672,"statfs":{"total":96636764160,"available":96606896128,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1319999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.044}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.087}]}]},{"osd":0,"up_from":162,"seq":695784701961,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34532,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337308,"statfs":{"total":96636764160,"available":96601403392,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.628 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.629 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.630 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T15:32:35.753712+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=35, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T15:32:29.715370+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=29, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.631 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.632 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.632 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.632 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:16.632 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T15:36:36.632 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:36.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:36.967 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-21T15:36:35.702238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15703,"ondisk_log_size":15703,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":344888,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":326565,"kb_avail":754629832,"statfs":{"total":773094113280,"available":772740947968,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":334403458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003644"},"pg_stats":[{"pgid":"3.1f","version":"181'671","reported_seq":5586,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:27.892547+0000","last_change":"2024-09-21T15:36:27.892757+0000","last_active":"2024-09-21T15:36:27.892547+0000","last_peered":"2024-09-21T15:36:27.892547+0000","last_clean":"2024-09-21T15:36:27.892547+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:27.892547+0000","last_undegraded":"2024-09-21T15:36:27.892547+0000","last_fullsized":"2024-09-21T15:36:27.892547+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'671","last_scrub_stamp":"2024-09-21T15:36:27.892492+0000","last_deep_scrub":"181'671","last_deep_scrub_stamp":"2024-09-21T15:36:27.892492+0000","last_clean_scrub_stamp":"2024-09-21T15:36:27.892492+0000","objects_scrubbed":5,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:56.367727+0000","scrub_duration":0.010697946999999999,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'509","reported_seq":7403,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:21.939250+0000","last_change":"2024-09-21T15:36:21.939498+0000","last_active":"2024-09-21T15:36:21.939250+0000","last_peered":"2024-09-21T15:36:21.939250+0000","last_clean":"2024-09-21T15:36:21.939250+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:21.939250+0000","last_undegraded":"2024-09-21T15:36:21.939250+0000","last_fullsized":"2024-09-21T15:36:21.939250+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'509","last_scrub_stamp":"2024-09-21T15:36:21.939196+0000","last_deep_scrub":"181'509","last_deep_scrub_stamp":"2024-09-21T15:36:21.939196+0000","last_clean_scrub_stamp":"2024-09-21T15:36:21.939196+0000","objects_scrubbed":4,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:30.144122+0000","scrub_duration":0.010358533,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'166","reported_seq":735,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.201631+0000","last_change":"2024-09-21T15:35:41.781777+0000","last_active":"2024-09-21T15:36:05.201631+0000","last_peered":"2024-09-21T15:36:05.201631+0000","last_clean":"2024-09-21T15:36:05.201631+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:05.201631+0000","last_undegraded":"2024-09-21T15:36:05.201631+0000","last_fullsized":"2024-09-21T15:36:05.201631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'333","reported_seq":6099,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.201302+0000","last_change":"2024-09-21T15:35:41.779788+0000","last_active":"2024-09-21T15:36:05.201302+0000","last_peered":"2024-09-21T15:36:05.201302+0000","last_clean":"2024-09-21T15:36:05.201302+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:05.201302+0000","last_undegraded":"2024-09-21T15:36:05.201302+0000","last_fullsized":"2024-09-21T15:36:05.201302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"181'441","reported_seq":10092,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.898756+0000","last_change":"2024-09-21T15:36:23.898972+0000","last_active":"2024-09-21T15:36:23.898756+0000","last_peered":"2024-09-21T15:36:23.898756+0000","last_clean":"2024-09-21T15:36:23.898756+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:23.898756+0000","last_undegraded":"2024-09-21T15:36:23.898756+0000","last_fullsized":"2024-09-21T15:36:23.898756+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'441","last_scrub_stamp":"2024-09-21T15:36:23.898696+0000","last_deep_scrub":"181'441","last_deep_scrub_stamp":"2024-09-21T15:36:23.898696+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.898696+0000","objects_scrubbed":14,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:15.780458+0000","scrub_duration":0.020266810999999999,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"181'274","reported_seq":7392,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.105026+0000","last_change":"2024-09-21T15:36:20.105240+0000","last_active":"2024-09-21T15:36:20.105026+0000","last_peered":"2024-09-21T15:36:20.105026+0000","last_clean":"2024-09-21T15:36:20.105026+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:20.105026+0000","last_undegraded":"2024-09-21T15:36:20.105026+0000","last_fullsized":"2024-09-21T15:36:20.105026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'274","last_scrub_stamp":"2024-09-21T15:36:20.104971+0000","last_deep_scrub":"181'274","last_deep_scrub_stamp":"2024-09-21T15:36:20.104971+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.104971+0000","objects_scrubbed":12,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:20:32.661540+0000","scrub_duration":0.018803362000000001,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"181'644","reported_seq":33626,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:14.281969+0000","last_change":"2024-09-21T15:36:14.282068+0000","last_active":"2024-09-21T15:36:14.281969+0000","last_peered":"2024-09-21T15:36:14.281969+0000","last_clean":"2024-09-21T15:36:14.281969+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:14.281969+0000","last_undegraded":"2024-09-21T15:36:14.281969+0000","last_fullsized":"2024-09-21T15:36:14.281969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'644","last_scrub_stamp":"2024-09-21T15:36:14.281943+0000","last_deep_scrub":"181'644","last_deep_scrub_stamp":"2024-09-21T15:36:14.281943+0000","last_clean_scrub_stamp":"2024-09-21T15:36:14.281943+0000","objects_scrubbed":7,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:06:48.691345+0000","scrub_duration":0.013465398,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'495","reported_seq":1381,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:14.975968+0000","last_change":"2024-09-21T15:36:14.976076+0000","last_active":"2024-09-21T15:36:14.975968+0000","last_peered":"2024-09-21T15:36:14.975968+0000","last_clean":"2024-09-21T15:36:14.975968+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:14.975968+0000","last_undegraded":"2024-09-21T15:36:14.975968+0000","last_fullsized":"2024-09-21T15:36:14.975968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'495","last_scrub_stamp":"2024-09-21T15:36:14.975942+0000","last_deep_scrub":"181'495","last_deep_scrub_stamp":"2024-09-21T15:36:14.975942+0000","last_clean_scrub_stamp":"2024-09-21T15:36:14.975942+0000","objects_scrubbed":5,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:05.650336+0000","scrub_duration":0.0057161629999999998,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"181'697","reported_seq":2428,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:30.888143+0000","last_change":"2024-09-21T15:36:30.888359+0000","last_active":"2024-09-21T15:36:30.888143+0000","last_peered":"2024-09-21T15:36:30.888143+0000","last_clean":"2024-09-21T15:36:30.888143+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:30.888143+0000","last_undegraded":"2024-09-21T15:36:30.888143+0000","last_fullsized":"2024-09-21T15:36:30.888143+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'697","last_scrub_stamp":"2024-09-21T15:36:30.888087+0000","last_deep_scrub":"181'697","last_deep_scrub_stamp":"2024-09-21T15:36:30.888087+0000","last_clean_scrub_stamp":"2024-09-21T15:36:30.888087+0000","objects_scrubbed":9,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:09.904576+0000","scrub_duration":0.016897946,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"181'375","reported_seq":6186,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:33.580444+0000","last_change":"2024-09-21T15:36:33.580698+0000","last_active":"2024-09-21T15:36:33.580444+0000","last_peered":"2024-09-21T15:36:33.580444+0000","last_clean":"2024-09-21T15:36:33.580444+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:33.580444+0000","last_undegraded":"2024-09-21T15:36:33.580444+0000","last_fullsized":"2024-09-21T15:36:33.580444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'375","last_scrub_stamp":"2024-09-21T15:36:33.580388+0000","last_deep_scrub":"181'375","last_deep_scrub_stamp":"2024-09-21T15:36:33.580388+0000","last_clean_scrub_stamp":"2024-09-21T15:36:33.580388+0000","objects_scrubbed":7,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:54.664578+0000","scrub_duration":0.014681779000000001,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'315","reported_seq":5335,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:33.550955+0000","last_change":"2024-09-21T15:36:33.551119+0000","last_active":"2024-09-21T15:36:33.550955+0000","last_peered":"2024-09-21T15:36:33.550955+0000","last_clean":"2024-09-21T15:36:33.550955+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:33.550955+0000","last_undegraded":"2024-09-21T15:36:33.550955+0000","last_fullsized":"2024-09-21T15:36:33.550955+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'315","last_scrub_stamp":"2024-09-21T15:36:33.550925+0000","last_deep_scrub":"181'315","last_deep_scrub_stamp":"2024-09-21T15:36:33.550925+0000","last_clean_scrub_stamp":"2024-09-21T15:36:33.550925+0000","objects_scrubbed":6,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:35.004717+0000","scrub_duration":0.011733525999999999,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"181'736","reported_seq":4727,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.179958+0000","last_change":"2024-09-21T15:36:16.180045+0000","last_active":"2024-09-21T15:36:16.179958+0000","last_peered":"2024-09-21T15:36:16.179958+0000","last_clean":"2024-09-21T15:36:16.179958+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:16.179958+0000","last_undegraded":"2024-09-21T15:36:16.179958+0000","last_fullsized":"2024-09-21T15:36:16.179958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'736","last_scrub_stamp":"2024-09-21T15:36:16.179937+0000","last_deep_scrub":"181'736","last_deep_scrub_stamp":"2024-09-21T15:36:16.179937+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.179937+0000","objects_scrubbed":10,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:18.662456+0000","scrub_duration":0.012936548000000001,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":203,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.780720+0000","last_change":"2024-09-21T15:36:16.780919+0000","last_active":"2024-09-21T15:36:16.780720+0000","last_peered":"2024-09-21T15:36:16.780720+0000","last_clean":"2024-09-21T15:36:16.780720+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:16.780720+0000","last_undegraded":"2024-09-21T15:36:16.780720+0000","last_fullsized":"2024-09-21T15:36:16.780720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:16.780671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:16.780671+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.780671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:27:42.571935+0000","scrub_duration":0.0028284069999999998,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"181'590","reported_seq":5146,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.898662+0000","last_change":"2024-09-21T15:36:22.898899+0000","last_active":"2024-09-21T15:36:22.898662+0000","last_peered":"2024-09-21T15:36:22.898662+0000","last_clean":"2024-09-21T15:36:22.898662+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:22.898662+0000","last_undegraded":"2024-09-21T15:36:22.898662+0000","last_fullsized":"2024-09-21T15:36:22.898662+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'590","last_scrub_stamp":"2024-09-21T15:36:22.898608+0000","last_deep_scrub":"181'590","last_deep_scrub_stamp":"2024-09-21T15:36:22.898608+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.898608+0000","objects_scrubbed":11,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:58.642056+0000","scrub_duration":0.021671355,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":179,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.346977+0000","last_change":"2024-09-21T15:36:23.347244+0000","last_active":"2024-09-21T15:36:23.346977+0000","last_peered":"2024-09-21T15:36:23.346977+0000","last_clean":"2024-09-21T15:36:23.346977+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:23.346977+0000","last_undegraded":"2024-09-21T15:36:23.346977+0000","last_fullsized":"2024-09-21T15:36:23.346977+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:23.346930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:23.346930+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.346930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:25.795673+0000","scrub_duration":0.003228884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"181'337","reported_seq":1706,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.927746+0000","last_change":"2024-09-21T15:36:23.927962+0000","last_active":"2024-09-21T15:36:23.927746+0000","last_peered":"2024-09-21T15:36:23.927746+0000","last_clean":"2024-09-21T15:36:23.927746+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:23.927746+0000","last_undegraded":"2024-09-21T15:36:23.927746+0000","last_fullsized":"2024-09-21T15:36:23.927746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'337","last_scrub_stamp":"2024-09-21T15:36:23.927692+0000","last_deep_scrub":"181'337","last_deep_scrub_stamp":"2024-09-21T15:36:23.927692+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.927692+0000","objects_scrubbed":12,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.365635+0000","scrub_duration":0.018494211999999999,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:17.455673+0000","last_change":"2024-09-21T15:36:17.455896+0000","last_active":"2024-09-21T15:36:17.455673+0000","last_peered":"2024-09-21T15:36:17.455673+0000","last_clean":"2024-09-21T15:36:17.455673+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:17.455673+0000","last_undegraded":"2024-09-21T15:36:17.455673+0000","last_fullsized":"2024-09-21T15:36:17.455673+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:17.455623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:17.455623+0000","last_clean_scrub_stamp":"2024-09-21T15:36:17.455623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:56:47.987946+0000","scrub_duration":0.0032200919999999999,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"181'241","reported_seq":1122,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:19.280757+0000","last_change":"2024-09-21T15:36:19.281004+0000","last_active":"2024-09-21T15:36:19.280757+0000","last_peered":"2024-09-21T15:36:19.280757+0000","last_clean":"2024-09-21T15:36:19.280757+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:19.280757+0000","last_undegraded":"2024-09-21T15:36:19.280757+0000","last_fullsized":"2024-09-21T15:36:19.280757+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'241","last_scrub_stamp":"2024-09-21T15:36:19.280704+0000","last_deep_scrub":"181'241","last_deep_scrub_stamp":"2024-09-21T15:36:19.280704+0000","last_clean_scrub_stamp":"2024-09-21T15:36:19.280704+0000","objects_scrubbed":9,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:12:14.770086+0000","scrub_duration":0.015914417,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:21.347160+0000","last_change":"2024-09-21T15:36:21.347356+0000","last_active":"2024-09-21T15:36:21.347160+0000","last_peered":"2024-09-21T15:36:21.347160+0000","last_clean":"2024-09-21T15:36:21.347160+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:21.347160+0000","last_undegraded":"2024-09-21T15:36:21.347160+0000","last_fullsized":"2024-09-21T15:36:21.347160+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:21.347112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:21.347112+0000","last_clean_scrub_stamp":"2024-09-21T15:36:21.347112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:06:04.293397+0000","scrub_duration":0.0028362190000000001,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'793","reported_seq":1576,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.682086+0000","last_change":"2024-09-21T15:36:29.682385+0000","last_active":"2024-09-21T15:36:29.682086+0000","last_peered":"2024-09-21T15:36:29.682086+0000","last_clean":"2024-09-21T15:36:29.682086+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:29.682086+0000","last_undegraded":"2024-09-21T15:36:29.682086+0000","last_fullsized":"2024-09-21T15:36:29.682086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'793","last_scrub_stamp":"2024-09-21T15:36:29.682034+0000","last_deep_scrub":"181'793","last_deep_scrub_stamp":"2024-09-21T15:36:29.682034+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.682034+0000","objects_scrubbed":10,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:23.950780+0000","scrub_duration":0.022368352000000001,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"181'460","reported_seq":7113,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:18.087458+0000","last_change":"2024-09-21T15:36:18.087705+0000","last_active":"2024-09-21T15:36:18.087458+0000","last_peered":"2024-09-21T15:36:18.087458+0000","last_clean":"2024-09-21T15:36:18.087458+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:18.087458+0000","last_undegraded":"2024-09-21T15:36:18.087458+0000","last_fullsized":"2024-09-21T15:36:18.087458+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'460","last_scrub_stamp":"2024-09-21T15:36:18.087400+0000","last_deep_scrub":"181'460","last_deep_scrub_stamp":"2024-09-21T15:36:18.087400+0000","last_clean_scrub_stamp":"2024-09-21T15:36:18.087400+0000","objects_scrubbed":10,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:19.643251+0000","scrub_duration":0.018600240000000001,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"181'3","reported_seq":191,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:24.649748+0000","last_change":"2024-09-21T15:36:24.650043+0000","last_active":"2024-09-21T15:36:24.649748+0000","last_peered":"2024-09-21T15:36:24.649748+0000","last_clean":"2024-09-21T15:36:24.649748+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:24.649748+0000","last_undegraded":"2024-09-21T15:36:24.649748+0000","last_fullsized":"2024-09-21T15:36:24.649748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'3","last_scrub_stamp":"2024-09-21T15:36:24.649698+0000","last_deep_scrub":"181'3","last_deep_scrub_stamp":"2024-09-21T15:36:24.649698+0000","last_clean_scrub_stamp":"2024-09-21T15:36:24.649698+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:21.480359+0000","scrub_duration":0.0068495860000000004,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'944","reported_seq":3014,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.386432+0000","last_change":"2024-09-21T15:36:22.386694+0000","last_active":"2024-09-21T15:36:22.386432+0000","last_peered":"2024-09-21T15:36:22.386432+0000","last_clean":"2024-09-21T15:36:22.386432+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:22.386432+0000","last_undegraded":"2024-09-21T15:36:22.386432+0000","last_fullsized":"2024-09-21T15:36:22.386432+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'944","last_scrub_stamp":"2024-09-21T15:36:22.386376+0000","last_deep_scrub":"181'944","last_deep_scrub_stamp":"2024-09-21T15:36:22.386376+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.386376+0000","objects_scrubbed":9,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:10.534271+0000","scrub_duration":0.019164569999999999,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:15.145895+0000","last_change":"2024-09-21T15:36:15.145990+0000","last_active":"2024-09-21T15:36:15.145895+0000","last_peered":"2024-09-21T15:36:15.145895+0000","last_clean":"2024-09-21T15:36:15.145895+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:15.145895+0000","last_undegraded":"2024-09-21T15:36:15.145895+0000","last_fullsized":"2024-09-21T15:36:15.145895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:15.145877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:15.145877+0000","last_clean_scrub_stamp":"2024-09-21T15:36:15.145877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:56.559743+0000","scrub_duration":0.001296361,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"181'539","reported_seq":3186,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.815016+0000","last_change":"2024-09-21T15:36:20.815266+0000","last_active":"2024-09-21T15:36:20.815016+0000","last_peered":"2024-09-21T15:36:20.815016+0000","last_clean":"2024-09-21T15:36:20.815016+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:20.815016+0000","last_undegraded":"2024-09-21T15:36:20.815016+0000","last_fullsized":"2024-09-21T15:36:20.815016+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'539","last_scrub_stamp":"2024-09-21T15:36:20.814953+0000","last_deep_scrub":"181'539","last_deep_scrub_stamp":"2024-09-21T15:36:20.814953+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.814953+0000","objects_scrubbed":6,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:11.324597+0000","scrub_duration":0.011672492,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":188,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.405582+0000","last_change":"2024-09-21T15:36:16.405836+0000","last_active":"2024-09-21T15:36:16.405582+0000","last_peered":"2024-09-21T15:36:16.405582+0000","last_clean":"2024-09-21T15:36:16.405582+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:16.405582+0000","last_undegraded":"2024-09-21T15:36:16.405582+0000","last_fullsized":"2024-09-21T15:36:16.405582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:16.405526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:16.405526+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.405526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:07.756016+0000","scrub_duration":0.002655517,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"181'420","reported_seq":1289,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:28.952470+0000","last_change":"2024-09-21T15:36:28.952742+0000","last_active":"2024-09-21T15:36:28.952470+0000","last_peered":"2024-09-21T15:36:28.952470+0000","last_clean":"2024-09-21T15:36:28.952470+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:28.952470+0000","last_undegraded":"2024-09-21T15:36:28.952470+0000","last_fullsized":"2024-09-21T15:36:28.952470+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'420","last_scrub_stamp":"2024-09-21T15:36:28.952415+0000","last_deep_scrub":"181'420","last_deep_scrub_stamp":"2024-09-21T15:36:28.952415+0000","last_clean_scrub_stamp":"2024-09-21T15:36:28.952415+0000","objects_scrubbed":15,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:21.845453+0000","scrub_duration":0.026168654999999999,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"181'2","reported_seq":214,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.605172+0000","last_change":"2024-09-21T15:36:22.605405+0000","last_active":"2024-09-21T15:36:22.605172+0000","last_peered":"2024-09-21T15:36:22.605172+0000","last_clean":"2024-09-21T15:36:22.605172+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:22.605172+0000","last_undegraded":"2024-09-21T15:36:22.605172+0000","last_fullsized":"2024-09-21T15:36:22.605172+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'2","last_scrub_stamp":"2024-09-21T15:36:22.605119+0000","last_deep_scrub":"181'2","last_deep_scrub_stamp":"2024-09-21T15:36:22.605119+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.605119+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:03.658340+0000","scrub_duration":0.0060498089999999997,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":355,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.298073+0000","last_change":"2024-09-21T15:35:53.779735+0000","last_active":"2024-09-21T15:36:16.298073+0000","last_peered":"2024-09-21T15:36:16.298073+0000","last_clean":"2024-09-21T15:36:16.298073+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:16.298073+0000","last_undegraded":"2024-09-21T15:36:16.298073+0000","last_fullsized":"2024-09-21T15:36:16.298073+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:29.715370+0000","last_clean_scrub_stamp":"2024-09-21T15:32:29.715370+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"181'623","reported_seq":1915,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.990688+0000","last_change":"2024-09-21T15:36:20.990944+0000","last_active":"2024-09-21T15:36:20.990688+0000","last_peered":"2024-09-21T15:36:20.990688+0000","last_clean":"2024-09-21T15:36:20.990688+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:20.990688+0000","last_undegraded":"2024-09-21T15:36:20.990688+0000","last_fullsized":"2024-09-21T15:36:20.990688+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'623","last_scrub_stamp":"2024-09-21T15:36:20.990634+0000","last_deep_scrub":"181'623","last_deep_scrub_stamp":"2024-09-21T15:36:20.990634+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.990634+0000","objects_scrubbed":12,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:24:55.421632+0000","scrub_duration":0.021719777999999999,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'558","reported_seq":1973,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:15.128396+0000","last_change":"2024-09-21T15:36:15.128635+0000","last_active":"2024-09-21T15:36:15.128396+0000","last_peered":"2024-09-21T15:36:15.128396+0000","last_clean":"2024-09-21T15:36:15.128396+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:15.128396+0000","last_undegraded":"2024-09-21T15:36:15.128396+0000","last_fullsized":"2024-09-21T15:36:15.128396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'558","last_scrub_stamp":"2024-09-21T15:36:15.128340+0000","last_deep_scrub":"181'558","last_deep_scrub_stamp":"2024-09-21T15:36:15.128340+0000","last_clean_scrub_stamp":"2024-09-21T15:36:15.128340+0000","objects_scrubbed":7,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:57:06.808712+0000","scrub_duration":0.013630486000000001,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"181'392","reported_seq":5212,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.799180+0000","last_change":"2024-09-21T15:36:29.799447+0000","last_active":"2024-09-21T15:36:29.799180+0000","last_peered":"2024-09-21T15:36:29.799180+0000","last_clean":"2024-09-21T15:36:29.799180+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:29.799180+0000","last_undegraded":"2024-09-21T15:36:29.799180+0000","last_fullsized":"2024-09-21T15:36:29.799180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'392","last_scrub_stamp":"2024-09-21T15:36:29.799122+0000","last_deep_scrub":"181'392","last_deep_scrub_stamp":"2024-09-21T15:36:29.799122+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.799122+0000","objects_scrubbed":8,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:23.079648+0000","scrub_duration":0.016157649999999999,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"159'429","reported_seq":9491,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.304309+0000","last_change":"2024-09-21T15:35:41.903624+0000","last_active":"2024-09-21T15:36:05.304309+0000","last_peered":"2024-09-21T15:36:05.304309+0000","last_clean":"2024-09-21T15:36:05.304309+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:05.304309+0000","last_undegraded":"2024-09-21T15:36:05.304309+0000","last_fullsized":"2024-09-21T15:36:05.304309+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"181'316","reported_seq":2326,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:31.661089+0000","last_change":"2024-09-21T15:36:31.661310+0000","last_active":"2024-09-21T15:36:31.661089+0000","last_peered":"2024-09-21T15:36:31.661089+0000","last_clean":"2024-09-21T15:36:31.661089+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:31.661089+0000","last_undegraded":"2024-09-21T15:36:31.661089+0000","last_fullsized":"2024-09-21T15:36:31.661089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'316","last_scrub_stamp":"2024-09-21T15:36:31.661037+0000","last_deep_scrub":"181'316","last_deep_scrub_stamp":"2024-09-21T15:36:31.661037+0000","last_clean_scrub_stamp":"2024-09-21T15:36:31.661037+0000","objects_scrubbed":4,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:55.846939+0000","scrub_duration":0.0097720120000000001,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"181'401","reported_seq":17642,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.561176+0000","last_change":"2024-09-21T15:36:23.561530+0000","last_active":"2024-09-21T15:36:23.561176+0000","last_peered":"2024-09-21T15:36:23.561176+0000","last_clean":"2024-09-21T15:36:23.561176+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:23.561176+0000","last_undegraded":"2024-09-21T15:36:23.561176+0000","last_fullsized":"2024-09-21T15:36:23.561176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'401","last_scrub_stamp":"2024-09-21T15:36:23.561125+0000","last_deep_scrub":"181'401","last_deep_scrub_stamp":"2024-09-21T15:36:23.561125+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.561125+0000","objects_scrubbed":6,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:12:01.327173+0000","scrub_duration":0.012493980999999999,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"181'381","reported_seq":38192,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:32.584334+0000","last_change":"2024-09-21T15:36:32.584580+0000","last_active":"2024-09-21T15:36:32.584334+0000","last_peered":"2024-09-21T15:36:32.584334+0000","last_clean":"2024-09-21T15:36:32.584334+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:32.584334+0000","last_undegraded":"2024-09-21T15:36:32.584334+0000","last_fullsized":"2024-09-21T15:36:32.584334+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'381","last_scrub_stamp":"2024-09-21T15:36:32.584279+0000","last_deep_scrub":"181'381","last_deep_scrub_stamp":"2024-09-21T15:36:32.584279+0000","last_clean_scrub_stamp":"2024-09-21T15:36:32.584279+0000","objects_scrubbed":5,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:11.771825+0000","scrub_duration":0.0126877,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'455","reported_seq":2090,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:05.304521+0000","last_change":"2024-09-21T15:35:41.906309+0000","last_active":"2024-09-21T15:36:05.304521+0000","last_peered":"2024-09-21T15:36:05.304521+0000","last_clean":"2024-09-21T15:36:05.304521+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:05.304521+0000","last_undegraded":"2024-09-21T15:36:05.304521+0000","last_fullsized":"2024-09-21T15:36:05.304521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:32:45.808046+0000","last_clean_scrub_stamp":"2024-09-21T15:32:45.808046+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"181'314","reported_seq":1530,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:30.800025+0000","last_change":"2024-09-21T15:36:30.800249+0000","last_active":"2024-09-21T15:36:30.800025+0000","last_peered":"2024-09-21T15:36:30.800025+0000","last_clean":"2024-09-21T15:36:30.800025+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:30.800025+0000","last_undegraded":"2024-09-21T15:36:30.800025+0000","last_fullsized":"2024-09-21T15:36:30.800025+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'314","last_scrub_stamp":"2024-09-21T15:36:30.799969+0000","last_deep_scrub":"181'314","last_deep_scrub_stamp":"2024-09-21T15:36:30.799969+0000","last_clean_scrub_stamp":"2024-09-21T15:36:30.799969+0000","objects_scrubbed":12,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:40.892369+0000","scrub_duration":0.018160714000000001,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"181'623","reported_seq":6853,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.536127+0000","last_change":"2024-09-21T15:36:22.536334+0000","last_active":"2024-09-21T15:36:22.536127+0000","last_peered":"2024-09-21T15:36:22.536127+0000","last_clean":"2024-09-21T15:36:22.536127+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:22.536127+0000","last_undegraded":"2024-09-21T15:36:22.536127+0000","last_fullsized":"2024-09-21T15:36:22.536127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'623","last_scrub_stamp":"2024-09-21T15:36:22.536072+0000","last_deep_scrub":"181'623","last_deep_scrub_stamp":"2024-09-21T15:36:22.536072+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.536072+0000","objects_scrubbed":4,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:55.126061+0000","scrub_duration":0.011084538,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"181'616","reported_seq":7864,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.905493+0000","last_change":"2024-09-21T15:36:29.905738+0000","last_active":"2024-09-21T15:36:29.905493+0000","last_peered":"2024-09-21T15:36:29.905493+0000","last_clean":"2024-09-21T15:36:29.905493+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:29.905493+0000","last_undegraded":"2024-09-21T15:36:29.905493+0000","last_fullsized":"2024-09-21T15:36:29.905493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'616","last_scrub_stamp":"2024-09-21T15:36:29.905433+0000","last_deep_scrub":"181'616","last_deep_scrub_stamp":"2024-09-21T15:36:29.905433+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.905433+0000","objects_scrubbed":4,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:20:10.182957+0000","scrub_duration":0.010341866999999999,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'493","reported_seq":11007,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:24.575997+0000","last_change":"2024-09-21T15:36:24.576220+0000","last_active":"2024-09-21T15:36:24.575997+0000","last_peered":"2024-09-21T15:36:24.575997+0000","last_clean":"2024-09-21T15:36:24.575997+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:24.575997+0000","last_undegraded":"2024-09-21T15:36:24.575997+0000","last_fullsized":"2024-09-21T15:36:24.575997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'493","last_scrub_stamp":"2024-09-21T15:36:24.575939+0000","last_deep_scrub":"181'493","last_deep_scrub_stamp":"2024-09-21T15:36:24.575939+0000","last_clean_scrub_stamp":"2024-09-21T15:36:24.575939+0000","objects_scrubbed":5,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:14.282186+0000","scrub_duration":0.011960442999999999,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15581,"ondisk_log_size":15581,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37920,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333920,"statfs":{"total":96636764160,"available":96597934080,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73299999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.94799999999999995,"5min":1.615,"15min":1.615},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68700000000000006,"15min":0.69499999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.2,"5min":1.268,"15min":1.268},"last":0.95799999999999996}]},{"osd":1,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.877,"15min":0.82299999999999995},"min":{"1min":0.40899999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.0620000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":1.181},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81599999999999995,"15min":0.85699999999999998},"min":{"1min":0.37,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0349999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0820000000000001,"5min":1.252,"15min":1.252},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.85599999999999998,"15min":0.875},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.1709999999999998,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":1.04}]},{"osd":3,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.88,"15min":0.81599999999999995},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1739999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.81000000000000005,"15min":0.86199999999999999},"min":{"1min":0.35199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3029999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.93899999999999995,"15min":0.875},"min":{"1min":0.36099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2490000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.91800000000000004}]},{"osd":5,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89300000000000002,"15min":0.94299999999999995},"min":{"1min":0.44,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3540000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.204},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89200000000000002,"15min":0.92100000000000004},"min":{"1min":0.46200000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.2969999999999999,"5min":1.365,"15min":1.365},"last":1.234}]},{"osd":6,"last update":"Sat Sep 21 15:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.94799999999999995,"15min":1},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4139999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.254},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0629999999999999,"15min":1.0269999999999999},"min":{"1min":0.498,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3380000000000001,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.276}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45532,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326308,"statfs":{"total":96636764160,"available":96590139392,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.82999999999999996,"15min":0.84799999999999998},"min":{"1min":0.40600000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1180000000000001,"5min":1.772,"15min":1.772},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.86599999999999999,"15min":0.89400000000000002},"min":{"1min":0.46999999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0609999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.85599999999999998}]},{"osd":1,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.82199999999999995,"15min":0.86599999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.163,"5min":1.452,"15min":1.452},"last":1.004}]},{"osd":2,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.82399999999999995,"15min":0.82699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":1.107}]},{"osd":3,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85799999999999998,"15min":0.874},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.85499999999999998,"15min":0.871},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.4079999999999999,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.93200000000000005,"15min":0.92600000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.6130000000000004,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.92100000000000004,"15min":0.93500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":3.5499999999999998,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.89000000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.022,"15min":1.0129999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.7170000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0089999999999999,"15min":1.0549999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4740000000000002,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":0.91600000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.79100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.111,"5min":1.2,"15min":1.2},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83999999999999997,"15min":0.85699999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0860000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.98699999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.76600000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":1.0760000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.82499999999999996,"15min":0.78500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80300000000000005,"15min":0.80600000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1559999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.92100000000000004,"15min":0.879},"min":{"1min":0.5,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1989999999999998,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":1.01},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.91600000000000004,"15min":0.91900000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0269999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.194,"5min":0.95299999999999996,"15min":0.92500000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4809999999999999,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.105},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92600000000000005,"15min":0.94099999999999995},"min":{"1min":0.45400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.93700000000000006}]},{"osd":7,"last update":"Sat Sep 21 15:35:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":0.996,"15min":0.98899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.214,"5min":2.214,"15min":2.214},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.107,"15min":1.0489999999999999},"min":{"1min":0.44900000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.5730000000000004,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41440,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330400,"statfs":{"total":96636764160,"available":96594329600,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77600000000000002,"15min":0.78300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0389999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1699999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.77500000000000002,"15min":0.80600000000000005},"min":{"1min":0.441,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.113,"5min":3.069,"15min":3.069},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.36499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0680000000000001,"5min":2.952,"15min":2.952},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.40500000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1299999999999999,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.129,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.93200000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.84399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.258,"5min":2.048,"15min":2.048},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.85699999999999998,"15min":0.88900000000000001},"min":{"1min":0.52200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.173,"5min":3.218,"15min":3.218},"last":0.77000000000000002}]},{"osd":5,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.83699999999999997,"15min":0.879},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.96699999999999997,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1619999999999999,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":1.1020000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.85699999999999998,"15min":0.876},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0249999999999999,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":1.048},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.99199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1990000000000001,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.0129999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.99099999999999999,"15min":1.0169999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.3839999999999999,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.60299999999999998,"15min":0.624},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78500000000000003}]},{"osd":1,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.61899999999999999},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4339999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61799999999999999,"15min":0.625},"min":{"1min":0.31900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.034,"5min":1.127,"15min":1.127},"last":0.81799999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.68899999999999995},"min":{"1min":0.33900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.042,"5min":1.802,"15min":1.802},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.71799999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2230000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.72999999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.57199999999999995,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.262,"5min":1.587,"15min":1.587},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.135,"15min":1.0149999999999999},"min":{"1min":0.60899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.143,"5min":24.064,"15min":24.064},"last":0.84999999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84699999999999998,"15min":0.91300000000000003},"min":{"1min":0.60599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.337,"5min":1.966,"15min":1.966},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.91500000000000004},"min":{"1min":0.58999999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.329,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.92800000000000005}]},{"osd":6,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86799999999999999,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.91800000000000004,"15min":0.96399999999999997},"min":{"1min":0.64600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5760000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.95099999999999996}]},{"osd":7,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93899999999999995,"15min":0.97699999999999998},"min":{"1min":0.54400000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.4359999999999999,"5min":1.772,"15min":1.772},"last":0.878},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93100000000000005,"15min":1.004},"min":{"1min":0.63700000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.397,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.159}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41436,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.60699999999999998,"15min":0.63300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.629}]},{"osd":1,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70799999999999996,"15min":0.751},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.157,"5min":1.613,"15min":1.613},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70399999999999996,"15min":0.754},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.107,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0660000000000001}]},{"osd":3,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.58699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0620000000000001,"5min":1.159,"15min":1.159},"last":0.79500000000000004}]},{"osd":4,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.90800000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.214,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88200000000000001,"15min":0.91200000000000003},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.103,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.133}]},{"osd":5,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.876},"min":{"1min":0.53800000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.236,"5min":1.536,"15min":1.536},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.89500000000000002,"15min":0.94499999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.179,"5min":1.52,"15min":1.52},"last":1.044}]},{"osd":6,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.91400000000000003,"15min":0.95999999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1359999999999999,"5min":1.617,"15min":1.617},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.421,"5min":1.554,"15min":1.554},"last":1.016}]},{"osd":7,"last update":"Sat Sep 21 15:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95199999999999996,"15min":0.98999999999999999},"min":{"1min":0.54800000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1970000000000001,"5min":1.669,"15min":1.669},"last":1.177},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.98799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.246,"5min":1.589,"15min":1.589},"last":1.218}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.56799999999999995,"15min":0.57999999999999996},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.79400000000000004,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.96899999999999997,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.28599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.0589999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.159,"5min":1.173,"15min":1.173},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.91000000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.79300000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.127,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.85799999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0960000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.995}]},{"osd":5,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.427,"5min":1.514,"15min":1.514},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":0.86899999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.365,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":1.0840000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.92600000000000005,"15min":0.95299999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.22,"5min":1.891,"15min":1.891},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.94199999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Sat Sep 21 15:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97199999999999998,"15min":0.998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.232,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.163}]}]},{"osd":0,"up_from":162,"seq":695784701965,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34528,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337312,"statfs":{"total":96636764160,"available":96601407488,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:36.970 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:36.970 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:36.971 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T15:32:29.715370+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=29, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:36.971 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:36.971 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T15:32:45.808046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=32, tm_sec=45, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=15, tm_min=36, tm_sec=11, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T15:36:36.971 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T15:36:56.973 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T15:36:57.287 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T15:36:57.288 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-21T15:36:57.304 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-21T15:36:55.708480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":253,"num_read":245645,"num_read_kb":504549,"num_write":17910,"num_write_kb":384184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":46085534,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15734,"ondisk_log_size":15734,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":348984,"kb_used_data":18232,"kb_used_omap":26,"kb_used_meta":330661,"kb_avail":754625736,"statfs":{"total":773094113280,"available":772736753664,"internally_reserved":0,"allocated":18669568,"data_stored":9587212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26750,"internal_metadata":338597762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003764"},"pg_stats":[{"pgid":"3.1f","version":"181'671","reported_seq":5586,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:27.892547+0000","last_change":"2024-09-21T15:36:27.892757+0000","last_active":"2024-09-21T15:36:27.892547+0000","last_peered":"2024-09-21T15:36:27.892547+0000","last_clean":"2024-09-21T15:36:27.892547+0000","last_became_active":"2024-09-21T15:35:53.781791+0000","last_became_peered":"2024-09-21T15:35:53.781791+0000","last_unstale":"2024-09-21T15:36:27.892547+0000","last_undegraded":"2024-09-21T15:36:27.892547+0000","last_fullsized":"2024-09-21T15:36:27.892547+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'671","last_scrub_stamp":"2024-09-21T15:36:27.892492+0000","last_deep_scrub":"181'671","last_deep_scrub_stamp":"2024-09-21T15:36:27.892492+0000","last_clean_scrub_stamp":"2024-09-21T15:36:27.892492+0000","objects_scrubbed":5,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:56.367727+0000","scrub_duration":0.010697946999999999,"objects_trimmed":0,"snaptrim_duration":0.002770303,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6986,"num_read_kb":8303,"num_write":986,"num_write_kb":3274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":1341665,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'509","reported_seq":7403,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:21.939250+0000","last_change":"2024-09-21T15:36:21.939498+0000","last_active":"2024-09-21T15:36:21.939250+0000","last_peered":"2024-09-21T15:36:21.939250+0000","last_clean":"2024-09-21T15:36:21.939250+0000","last_became_active":"2024-09-21T15:35:55.788783+0000","last_became_peered":"2024-09-21T15:35:55.788783+0000","last_unstale":"2024-09-21T15:36:21.939250+0000","last_undegraded":"2024-09-21T15:36:21.939250+0000","last_fullsized":"2024-09-21T15:36:21.939250+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'509","last_scrub_stamp":"2024-09-21T15:36:21.939196+0000","last_deep_scrub":"181'509","last_deep_scrub_stamp":"2024-09-21T15:36:21.939196+0000","last_clean_scrub_stamp":"2024-09-21T15:36:21.939196+0000","objects_scrubbed":4,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:56:30.144122+0000","scrub_duration":0.010358533,"objects_trimmed":0,"snaptrim_duration":0.00056735100000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8802,"num_read_kb":21884,"num_write":606,"num_write_kb":26515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"181'172","reported_seq":752,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:36.583063+0000","last_change":"2024-09-21T15:36:36.583288+0000","last_active":"2024-09-21T15:36:36.583063+0000","last_peered":"2024-09-21T15:36:36.583063+0000","last_clean":"2024-09-21T15:36:36.583063+0000","last_became_active":"2024-09-21T15:32:46.732592+0000","last_became_peered":"2024-09-21T15:32:46.732592+0000","last_unstale":"2024-09-21T15:36:36.583063+0000","last_undegraded":"2024-09-21T15:36:36.583063+0000","last_fullsized":"2024-09-21T15:36:36.583063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'172","last_scrub_stamp":"2024-09-21T15:36:36.583010+0000","last_deep_scrub":"181'172","last_deep_scrub_stamp":"2024-09-21T15:36:36.583010+0000","last_clean_scrub_stamp":"2024-09-21T15:36:36.583010+0000","objects_scrubbed":6,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:02:13.314057+0000","scrub_duration":0.012112335,"objects_trimmed":0,"snaptrim_duration":0.0049742359999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":293,"num_read_kb":6251,"num_write":212,"num_write_kb":25417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"181'342","reported_seq":6119,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:35.583014+0000","last_change":"2024-09-21T15:36:35.583254+0000","last_active":"2024-09-21T15:36:35.583014+0000","last_peered":"2024-09-21T15:36:35.583014+0000","last_clean":"2024-09-21T15:36:35.583014+0000","last_became_active":"2024-09-21T15:32:46.735007+0000","last_became_peered":"2024-09-21T15:32:46.735007+0000","last_unstale":"2024-09-21T15:36:35.583014+0000","last_undegraded":"2024-09-21T15:36:35.583014+0000","last_fullsized":"2024-09-21T15:36:35.583014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'342","last_scrub_stamp":"2024-09-21T15:36:35.582962+0000","last_deep_scrub":"181'342","last_deep_scrub_stamp":"2024-09-21T15:36:35.582962+0000","last_clean_scrub_stamp":"2024-09-21T15:36:35.582962+0000","objects_scrubbed":9,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:19:34.161126+0000","scrub_duration":0.016432585,"objects_trimmed":0,"snaptrim_duration":0.003566802,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":5734,"num_read_kb":23150,"num_write":364,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"181'441","reported_seq":10092,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.898756+0000","last_change":"2024-09-21T15:36:23.898972+0000","last_active":"2024-09-21T15:36:23.898756+0000","last_peered":"2024-09-21T15:36:23.898756+0000","last_clean":"2024-09-21T15:36:23.898756+0000","last_became_active":"2024-09-21T15:35:52.773514+0000","last_became_peered":"2024-09-21T15:35:52.773514+0000","last_unstale":"2024-09-21T15:36:23.898756+0000","last_undegraded":"2024-09-21T15:36:23.898756+0000","last_fullsized":"2024-09-21T15:36:23.898756+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'441","last_scrub_stamp":"2024-09-21T15:36:23.898696+0000","last_deep_scrub":"181'441","last_deep_scrub_stamp":"2024-09-21T15:36:23.898696+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.898696+0000","objects_scrubbed":14,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:15.780458+0000","scrub_duration":0.020266810999999999,"objects_trimmed":0,"snaptrim_duration":0.0037876590000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10419,"num_read_kb":17497,"num_write":387,"num_write_kb":7287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":1858814,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"181'274","reported_seq":7392,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.105026+0000","last_change":"2024-09-21T15:36:20.105240+0000","last_active":"2024-09-21T15:36:20.105026+0000","last_peered":"2024-09-21T15:36:20.105026+0000","last_clean":"2024-09-21T15:36:20.105026+0000","last_became_active":"2024-09-21T15:32:46.736464+0000","last_became_peered":"2024-09-21T15:32:46.736464+0000","last_unstale":"2024-09-21T15:36:20.105026+0000","last_undegraded":"2024-09-21T15:36:20.105026+0000","last_fullsized":"2024-09-21T15:36:20.105026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'274","last_scrub_stamp":"2024-09-21T15:36:20.104971+0000","last_deep_scrub":"181'274","last_deep_scrub_stamp":"2024-09-21T15:36:20.104971+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.104971+0000","objects_scrubbed":12,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:20:32.661540+0000","scrub_duration":0.018803362000000001,"objects_trimmed":0,"snaptrim_duration":0.0016161260000000001,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":6886,"num_read_kb":36085,"num_write":233,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"181'644","reported_seq":33626,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:14.281969+0000","last_change":"2024-09-21T15:36:14.282068+0000","last_active":"2024-09-21T15:36:14.281969+0000","last_peered":"2024-09-21T15:36:14.281969+0000","last_clean":"2024-09-21T15:36:14.281969+0000","last_became_active":"2024-09-21T15:32:46.731573+0000","last_became_peered":"2024-09-21T15:32:46.731573+0000","last_unstale":"2024-09-21T15:36:14.281969+0000","last_undegraded":"2024-09-21T15:36:14.281969+0000","last_fullsized":"2024-09-21T15:36:14.281969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'644","last_scrub_stamp":"2024-09-21T15:36:14.281943+0000","last_deep_scrub":"181'644","last_deep_scrub_stamp":"2024-09-21T15:36:14.281943+0000","last_clean_scrub_stamp":"2024-09-21T15:36:14.281943+0000","objects_scrubbed":7,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:06:48.691345+0000","scrub_duration":0.013465398,"objects_trimmed":0,"snaptrim_duration":0.00063854600000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":34681,"num_read_kb":22778,"num_write":849,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"181'495","reported_seq":1381,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:14.975968+0000","last_change":"2024-09-21T15:36:14.976076+0000","last_active":"2024-09-21T15:36:14.975968+0000","last_peered":"2024-09-21T15:36:14.975968+0000","last_clean":"2024-09-21T15:36:14.975968+0000","last_became_active":"2024-09-21T15:35:55.783241+0000","last_became_peered":"2024-09-21T15:35:55.783241+0000","last_unstale":"2024-09-21T15:36:14.975968+0000","last_undegraded":"2024-09-21T15:36:14.975968+0000","last_fullsized":"2024-09-21T15:36:14.975968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'495","last_scrub_stamp":"2024-09-21T15:36:14.975942+0000","last_deep_scrub":"181'495","last_deep_scrub_stamp":"2024-09-21T15:36:14.975942+0000","last_clean_scrub_stamp":"2024-09-21T15:36:14.975942+0000","objects_scrubbed":5,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:07:05.650336+0000","scrub_duration":0.0057161629999999998,"objects_trimmed":0,"snaptrim_duration":0.00044185300000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":2355,"num_read_kb":4936,"num_write":635,"num_write_kb":13282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":919033,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"181'697","reported_seq":2428,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:30.888143+0000","last_change":"2024-09-21T15:36:30.888359+0000","last_active":"2024-09-21T15:36:30.888143+0000","last_peered":"2024-09-21T15:36:30.888143+0000","last_clean":"2024-09-21T15:36:30.888143+0000","last_became_active":"2024-09-21T15:35:55.784417+0000","last_became_peered":"2024-09-21T15:35:55.784417+0000","last_unstale":"2024-09-21T15:36:30.888143+0000","last_undegraded":"2024-09-21T15:36:30.888143+0000","last_fullsized":"2024-09-21T15:36:30.888143+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'697","last_scrub_stamp":"2024-09-21T15:36:30.888087+0000","last_deep_scrub":"181'697","last_deep_scrub_stamp":"2024-09-21T15:36:30.888087+0000","last_clean_scrub_stamp":"2024-09-21T15:36:30.888087+0000","objects_scrubbed":9,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:09.904576+0000","scrub_duration":0.016897946,"objects_trimmed":0,"snaptrim_duration":0.0035472469999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4165,"num_read_kb":11727,"num_write":978,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2845882,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"181'375","reported_seq":6186,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:33.580444+0000","last_change":"2024-09-21T15:36:33.580698+0000","last_active":"2024-09-21T15:36:33.580444+0000","last_peered":"2024-09-21T15:36:33.580444+0000","last_clean":"2024-09-21T15:36:33.580444+0000","last_became_active":"2024-09-21T15:32:46.729830+0000","last_became_peered":"2024-09-21T15:32:46.729830+0000","last_unstale":"2024-09-21T15:36:33.580444+0000","last_undegraded":"2024-09-21T15:36:33.580444+0000","last_fullsized":"2024-09-21T15:36:33.580444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'375","last_scrub_stamp":"2024-09-21T15:36:33.580388+0000","last_deep_scrub":"181'375","last_deep_scrub_stamp":"2024-09-21T15:36:33.580388+0000","last_clean_scrub_stamp":"2024-09-21T15:36:33.580388+0000","objects_scrubbed":7,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:54.664578+0000","scrub_duration":0.014681779000000001,"objects_trimmed":0,"snaptrim_duration":0.0043691900000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7143,"num_read_kb":11614,"num_write":343,"num_write_kb":5072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'315","reported_seq":5335,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:33.550955+0000","last_change":"2024-09-21T15:36:33.551119+0000","last_active":"2024-09-21T15:36:33.550955+0000","last_peered":"2024-09-21T15:36:33.550955+0000","last_clean":"2024-09-21T15:36:33.550955+0000","last_became_active":"2024-09-21T15:34:33.799754+0000","last_became_peered":"2024-09-21T15:34:33.799754+0000","last_unstale":"2024-09-21T15:36:33.550955+0000","last_undegraded":"2024-09-21T15:36:33.550955+0000","last_fullsized":"2024-09-21T15:36:33.550955+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'315","last_scrub_stamp":"2024-09-21T15:36:33.550925+0000","last_deep_scrub":"181'315","last_deep_scrub_stamp":"2024-09-21T15:36:33.550925+0000","last_clean_scrub_stamp":"2024-09-21T15:36:33.550925+0000","objects_scrubbed":6,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:37:35.004717+0000","scrub_duration":0.011733525999999999,"objects_trimmed":0,"snaptrim_duration":0.0048302880000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4863,"num_read_kb":14923,"num_write":336,"num_write_kb":13347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":2440967,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"181'736","reported_seq":4728,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.180045+0000","last_change":"2024-09-21T15:36:16.180045+0000","last_active":"2024-09-21T15:36:16.180045+0000","last_peered":"2024-09-21T15:36:16.180045+0000","last_clean":"2024-09-21T15:36:16.180045+0000","last_became_active":"2024-09-21T15:34:22.149196+0000","last_became_peered":"2024-09-21T15:34:22.149196+0000","last_unstale":"2024-09-21T15:36:16.180045+0000","last_undegraded":"2024-09-21T15:36:16.180045+0000","last_fullsized":"2024-09-21T15:36:16.180045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'736","last_scrub_stamp":"2024-09-21T15:36:16.179937+0000","last_deep_scrub":"181'736","last_deep_scrub_stamp":"2024-09-21T15:36:16.179937+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.179937+0000","objects_scrubbed":10,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:18.662456+0000","scrub_duration":0.012936548000000001,"objects_trimmed":0,"snaptrim_duration":0.001221766,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6587,"num_read_kb":16022,"num_write":840,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1193404,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":203,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.780720+0000","last_change":"2024-09-21T15:36:16.780919+0000","last_active":"2024-09-21T15:36:16.780720+0000","last_peered":"2024-09-21T15:36:16.780720+0000","last_clean":"2024-09-21T15:36:16.780720+0000","last_became_active":"2024-09-21T15:34:30.769576+0000","last_became_peered":"2024-09-21T15:34:30.769576+0000","last_unstale":"2024-09-21T15:36:16.780720+0000","last_undegraded":"2024-09-21T15:36:16.780720+0000","last_fullsized":"2024-09-21T15:36:16.780720+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:16.780671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:16.780671+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.780671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:27:42.571935+0000","scrub_duration":0.0028284069999999998,"objects_trimmed":0,"snaptrim_duration":0.00065521100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"181'590","reported_seq":5146,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.898662+0000","last_change":"2024-09-21T15:36:22.898899+0000","last_active":"2024-09-21T15:36:22.898662+0000","last_peered":"2024-09-21T15:36:22.898662+0000","last_clean":"2024-09-21T15:36:22.898662+0000","last_became_active":"2024-09-21T15:35:52.773611+0000","last_became_peered":"2024-09-21T15:35:52.773611+0000","last_unstale":"2024-09-21T15:36:22.898662+0000","last_undegraded":"2024-09-21T15:36:22.898662+0000","last_fullsized":"2024-09-21T15:36:22.898662+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'590","last_scrub_stamp":"2024-09-21T15:36:22.898608+0000","last_deep_scrub":"181'590","last_deep_scrub_stamp":"2024-09-21T15:36:22.898608+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.898608+0000","objects_scrubbed":11,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:41:58.642056+0000","scrub_duration":0.021671355,"objects_trimmed":0,"snaptrim_duration":0.0028757119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7756,"num_read_kb":14010,"num_write":469,"num_write_kb":4724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":1283110,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":179,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.346977+0000","last_change":"2024-09-21T15:36:23.347244+0000","last_active":"2024-09-21T15:36:23.346977+0000","last_peered":"2024-09-21T15:36:23.346977+0000","last_clean":"2024-09-21T15:36:23.346977+0000","last_became_active":"2024-09-21T15:35:52.752179+0000","last_became_peered":"2024-09-21T15:35:52.752179+0000","last_unstale":"2024-09-21T15:36:23.346977+0000","last_undegraded":"2024-09-21T15:36:23.346977+0000","last_fullsized":"2024-09-21T15:36:23.346977+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:23.346930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:23.346930+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.346930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:25.795673+0000","scrub_duration":0.003228884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"181'337","reported_seq":1706,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.927746+0000","last_change":"2024-09-21T15:36:23.927962+0000","last_active":"2024-09-21T15:36:23.927746+0000","last_peered":"2024-09-21T15:36:23.927746+0000","last_clean":"2024-09-21T15:36:23.927746+0000","last_became_active":"2024-09-21T15:35:55.788830+0000","last_became_peered":"2024-09-21T15:35:55.788830+0000","last_unstale":"2024-09-21T15:36:23.927746+0000","last_undegraded":"2024-09-21T15:36:23.927746+0000","last_fullsized":"2024-09-21T15:36:23.927746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'337","last_scrub_stamp":"2024-09-21T15:36:23.927692+0000","last_deep_scrub":"181'337","last_deep_scrub_stamp":"2024-09-21T15:36:23.927692+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.927692+0000","objects_scrubbed":12,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.365635+0000","scrub_duration":0.018494211999999999,"objects_trimmed":0,"snaptrim_duration":0.0030148480000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1901,"num_read_kb":23302,"num_write":370,"num_write_kb":18395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:17.455896+0000","last_change":"2024-09-21T15:36:17.455896+0000","last_active":"2024-09-21T15:36:17.455896+0000","last_peered":"2024-09-21T15:36:17.455896+0000","last_clean":"2024-09-21T15:36:17.455896+0000","last_became_active":"2024-09-21T15:35:50.058715+0000","last_became_peered":"2024-09-21T15:35:50.058715+0000","last_unstale":"2024-09-21T15:36:17.455896+0000","last_undegraded":"2024-09-21T15:36:17.455896+0000","last_fullsized":"2024-09-21T15:36:17.455896+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:17.455623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:17.455623+0000","last_clean_scrub_stamp":"2024-09-21T15:36:17.455623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:56:47.987946+0000","scrub_duration":0.0032200919999999999,"objects_trimmed":0,"snaptrim_duration":0.00071484399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"181'241","reported_seq":1122,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:19.280757+0000","last_change":"2024-09-21T15:36:19.281004+0000","last_active":"2024-09-21T15:36:19.280757+0000","last_peered":"2024-09-21T15:36:19.280757+0000","last_clean":"2024-09-21T15:36:19.280757+0000","last_became_active":"2024-09-21T15:35:51.776158+0000","last_became_peered":"2024-09-21T15:35:51.776158+0000","last_unstale":"2024-09-21T15:36:19.280757+0000","last_undegraded":"2024-09-21T15:36:19.280757+0000","last_fullsized":"2024-09-21T15:36:19.280757+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'241","last_scrub_stamp":"2024-09-21T15:36:19.280704+0000","last_deep_scrub":"181'241","last_deep_scrub_stamp":"2024-09-21T15:36:19.280704+0000","last_clean_scrub_stamp":"2024-09-21T15:36:19.280704+0000","objects_scrubbed":9,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:12:14.770086+0000","scrub_duration":0.015914417,"objects_trimmed":0,"snaptrim_duration":0.001100597,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1042,"num_read_kb":4252,"num_write":221,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":67566,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:21.347356+0000","last_change":"2024-09-21T15:36:21.347356+0000","last_active":"2024-09-21T15:36:21.347356+0000","last_peered":"2024-09-21T15:36:21.347356+0000","last_clean":"2024-09-21T15:36:21.347356+0000","last_became_active":"2024-09-21T15:35:51.778009+0000","last_became_peered":"2024-09-21T15:35:51.778009+0000","last_unstale":"2024-09-21T15:36:21.347356+0000","last_undegraded":"2024-09-21T15:36:21.347356+0000","last_fullsized":"2024-09-21T15:36:21.347356+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:21.347112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:21.347112+0000","last_clean_scrub_stamp":"2024-09-21T15:36:21.347112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:06:04.293397+0000","scrub_duration":0.0028362190000000001,"objects_trimmed":0,"snaptrim_duration":0.001195614,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"181'793","reported_seq":1576,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.682086+0000","last_change":"2024-09-21T15:36:29.682385+0000","last_active":"2024-09-21T15:36:29.682086+0000","last_peered":"2024-09-21T15:36:29.682086+0000","last_clean":"2024-09-21T15:36:29.682086+0000","last_became_active":"2024-09-21T15:32:46.734504+0000","last_became_peered":"2024-09-21T15:32:46.734504+0000","last_unstale":"2024-09-21T15:36:29.682086+0000","last_undegraded":"2024-09-21T15:36:29.682086+0000","last_fullsized":"2024-09-21T15:36:29.682086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'793","last_scrub_stamp":"2024-09-21T15:36:29.682034+0000","last_deep_scrub":"181'793","last_deep_scrub_stamp":"2024-09-21T15:36:29.682034+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.682034+0000","objects_scrubbed":10,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:23.950780+0000","scrub_duration":0.022368352000000001,"objects_trimmed":0,"snaptrim_duration":0.003392876,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3240,"num_read_kb":10618,"num_write":1068,"num_write_kb":5999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"181'460","reported_seq":7113,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:18.087458+0000","last_change":"2024-09-21T15:36:18.087705+0000","last_active":"2024-09-21T15:36:18.087458+0000","last_peered":"2024-09-21T15:36:18.087458+0000","last_clean":"2024-09-21T15:36:18.087458+0000","last_became_active":"2024-09-21T15:35:55.784458+0000","last_became_peered":"2024-09-21T15:35:55.784458+0000","last_unstale":"2024-09-21T15:36:18.087458+0000","last_undegraded":"2024-09-21T15:36:18.087458+0000","last_fullsized":"2024-09-21T15:36:18.087458+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'460","last_scrub_stamp":"2024-09-21T15:36:18.087400+0000","last_deep_scrub":"181'460","last_deep_scrub_stamp":"2024-09-21T15:36:18.087400+0000","last_clean_scrub_stamp":"2024-09-21T15:36:18.087400+0000","objects_scrubbed":10,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:19.643251+0000","scrub_duration":0.018600240000000001,"objects_trimmed":0,"snaptrim_duration":0.0032182629999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6087,"num_read_kb":16626,"num_write":444,"num_write_kb":6851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":1771021,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"181'3","reported_seq":192,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:24.650043+0000","last_change":"2024-09-21T15:36:24.650043+0000","last_active":"2024-09-21T15:36:24.650043+0000","last_peered":"2024-09-21T15:36:24.650043+0000","last_clean":"2024-09-21T15:36:24.650043+0000","last_became_active":"2024-09-21T15:32:36.792284+0000","last_became_peered":"2024-09-21T15:32:36.792284+0000","last_unstale":"2024-09-21T15:36:24.650043+0000","last_undegraded":"2024-09-21T15:36:24.650043+0000","last_fullsized":"2024-09-21T15:36:24.650043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'3","last_scrub_stamp":"2024-09-21T15:36:24.649698+0000","last_deep_scrub":"181'3","last_deep_scrub_stamp":"2024-09-21T15:36:24.649698+0000","last_clean_scrub_stamp":"2024-09-21T15:36:24.649698+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:21.480359+0000","scrub_duration":0.0068495860000000004,"objects_trimmed":0,"snaptrim_duration":0.0018069589999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"181'944","reported_seq":3015,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.386694+0000","last_change":"2024-09-21T15:36:22.386694+0000","last_active":"2024-09-21T15:36:22.386694+0000","last_peered":"2024-09-21T15:36:22.386694+0000","last_clean":"2024-09-21T15:36:22.386694+0000","last_became_active":"2024-09-21T15:35:51.775804+0000","last_became_peered":"2024-09-21T15:35:51.775804+0000","last_unstale":"2024-09-21T15:36:22.386694+0000","last_undegraded":"2024-09-21T15:36:22.386694+0000","last_fullsized":"2024-09-21T15:36:22.386694+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'944","last_scrub_stamp":"2024-09-21T15:36:22.386376+0000","last_deep_scrub":"181'944","last_deep_scrub_stamp":"2024-09-21T15:36:22.386376+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.386376+0000","objects_scrubbed":9,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:10.534271+0000","scrub_duration":0.019164569999999999,"objects_trimmed":0,"snaptrim_duration":0.0066993069999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6269,"num_read_kb":9448,"num_write":1183,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":399106,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:15.145895+0000","last_change":"2024-09-21T15:36:15.145990+0000","last_active":"2024-09-21T15:36:15.145895+0000","last_peered":"2024-09-21T15:36:15.145895+0000","last_clean":"2024-09-21T15:36:15.145895+0000","last_became_active":"2024-09-21T15:33:21.065716+0000","last_became_peered":"2024-09-21T15:33:21.065716+0000","last_unstale":"2024-09-21T15:36:15.145895+0000","last_undegraded":"2024-09-21T15:36:15.145895+0000","last_fullsized":"2024-09-21T15:36:15.145895+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:15.145877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:15.145877+0000","last_clean_scrub_stamp":"2024-09-21T15:36:15.145877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:56.559743+0000","scrub_duration":0.001296361,"objects_trimmed":0,"snaptrim_duration":0.00029796100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"181'539","reported_seq":3186,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.815016+0000","last_change":"2024-09-21T15:36:20.815266+0000","last_active":"2024-09-21T15:36:20.815016+0000","last_peered":"2024-09-21T15:36:20.815016+0000","last_clean":"2024-09-21T15:36:20.815016+0000","last_became_active":"2024-09-21T15:35:52.776875+0000","last_became_peered":"2024-09-21T15:35:52.776875+0000","last_unstale":"2024-09-21T15:36:20.815016+0000","last_undegraded":"2024-09-21T15:36:20.815016+0000","last_fullsized":"2024-09-21T15:36:20.815016+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'539","last_scrub_stamp":"2024-09-21T15:36:20.814953+0000","last_deep_scrub":"181'539","last_deep_scrub_stamp":"2024-09-21T15:36:20.814953+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.814953+0000","objects_scrubbed":6,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:11.324597+0000","scrub_duration":0.011672492,"objects_trimmed":1,"snaptrim_duration":0.0039914529999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4099,"num_read_kb":12523,"num_write":707,"num_write_kb":13941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":3827628,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:16.405836+0000","last_change":"2024-09-21T15:36:16.405836+0000","last_active":"2024-09-21T15:36:16.405836+0000","last_peered":"2024-09-21T15:36:16.405836+0000","last_clean":"2024-09-21T15:36:16.405836+0000","last_became_active":"2024-09-21T15:32:36.792518+0000","last_became_peered":"2024-09-21T15:32:36.792518+0000","last_unstale":"2024-09-21T15:36:16.405836+0000","last_undegraded":"2024-09-21T15:36:16.405836+0000","last_fullsized":"2024-09-21T15:36:16.405836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T15:36:16.405526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T15:36:16.405526+0000","last_clean_scrub_stamp":"2024-09-21T15:36:16.405526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:51:07.756016+0000","scrub_duration":0.002655517,"objects_trimmed":0,"snaptrim_duration":0.00068160200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"181'420","reported_seq":1289,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:28.952470+0000","last_change":"2024-09-21T15:36:28.952742+0000","last_active":"2024-09-21T15:36:28.952470+0000","last_peered":"2024-09-21T15:36:28.952470+0000","last_clean":"2024-09-21T15:36:28.952470+0000","last_became_active":"2024-09-21T15:35:55.783766+0000","last_became_peered":"2024-09-21T15:35:55.783766+0000","last_unstale":"2024-09-21T15:36:28.952470+0000","last_undegraded":"2024-09-21T15:36:28.952470+0000","last_fullsized":"2024-09-21T15:36:28.952470+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'420","last_scrub_stamp":"2024-09-21T15:36:28.952415+0000","last_deep_scrub":"181'420","last_deep_scrub_stamp":"2024-09-21T15:36:28.952415+0000","last_clean_scrub_stamp":"2024-09-21T15:36:28.952415+0000","objects_scrubbed":15,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:21.845453+0000","scrub_duration":0.026168654999999999,"objects_trimmed":0,"snaptrim_duration":0.00071626999999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1752,"num_read_kb":4929,"num_write":419,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"181'2","reported_seq":215,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.605405+0000","last_change":"2024-09-21T15:36:22.605405+0000","last_active":"2024-09-21T15:36:22.605405+0000","last_peered":"2024-09-21T15:36:22.605405+0000","last_clean":"2024-09-21T15:36:22.605405+0000","last_became_active":"2024-09-21T15:33:21.067294+0000","last_became_peered":"2024-09-21T15:33:21.067294+0000","last_unstale":"2024-09-21T15:36:22.605405+0000","last_undegraded":"2024-09-21T15:36:22.605405+0000","last_fullsized":"2024-09-21T15:36:22.605405+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'2","last_scrub_stamp":"2024-09-21T15:36:22.605119+0000","last_deep_scrub":"181'2","last_deep_scrub_stamp":"2024-09-21T15:36:22.605119+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.605119+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:03.658340+0000","scrub_duration":0.0060498089999999997,"objects_trimmed":0,"snaptrim_duration":0.00080393999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"181'119","reported_seq":368,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:38.644403+0000","last_change":"2024-09-21T15:36:38.644613+0000","last_active":"2024-09-21T15:36:38.644403+0000","last_peered":"2024-09-21T15:36:38.644403+0000","last_clean":"2024-09-21T15:36:38.644403+0000","last_became_active":"2024-09-21T15:35:53.778963+0000","last_became_peered":"2024-09-21T15:35:53.778963+0000","last_unstale":"2024-09-21T15:36:38.644403+0000","last_undegraded":"2024-09-21T15:36:38.644403+0000","last_fullsized":"2024-09-21T15:36:38.644403+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'119","last_scrub_stamp":"2024-09-21T15:36:38.644350+0000","last_deep_scrub":"181'119","last_deep_scrub_stamp":"2024-09-21T15:36:38.644350+0000","last_clean_scrub_stamp":"2024-09-21T15:36:38.644350+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:06:05.578008+0000","scrub_duration":0.0090995720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"181'623","reported_seq":1915,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:20.990688+0000","last_change":"2024-09-21T15:36:20.990944+0000","last_active":"2024-09-21T15:36:20.990688+0000","last_peered":"2024-09-21T15:36:20.990688+0000","last_clean":"2024-09-21T15:36:20.990688+0000","last_became_active":"2024-09-21T15:35:55.784955+0000","last_became_peered":"2024-09-21T15:35:55.784955+0000","last_unstale":"2024-09-21T15:36:20.990688+0000","last_undegraded":"2024-09-21T15:36:20.990688+0000","last_fullsized":"2024-09-21T15:36:20.990688+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'623","last_scrub_stamp":"2024-09-21T15:36:20.990634+0000","last_deep_scrub":"181'623","last_deep_scrub_stamp":"2024-09-21T15:36:20.990634+0000","last_clean_scrub_stamp":"2024-09-21T15:36:20.990634+0000","objects_scrubbed":12,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:24:55.421632+0000","scrub_duration":0.021719777999999999,"objects_trimmed":0,"snaptrim_duration":0.00051575899999999997,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3173,"num_read_kb":7307,"num_write":825,"num_write_kb":5834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3093631,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'558","reported_seq":1973,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:15.128396+0000","last_change":"2024-09-21T15:36:15.128635+0000","last_active":"2024-09-21T15:36:15.128396+0000","last_peered":"2024-09-21T15:36:15.128396+0000","last_clean":"2024-09-21T15:36:15.128396+0000","last_became_active":"2024-09-21T15:33:21.067657+0000","last_became_peered":"2024-09-21T15:33:21.067657+0000","last_unstale":"2024-09-21T15:36:15.128396+0000","last_undegraded":"2024-09-21T15:36:15.128396+0000","last_fullsized":"2024-09-21T15:36:15.128396+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'558","last_scrub_stamp":"2024-09-21T15:36:15.128340+0000","last_deep_scrub":"181'558","last_deep_scrub_stamp":"2024-09-21T15:36:15.128340+0000","last_clean_scrub_stamp":"2024-09-21T15:36:15.128340+0000","objects_scrubbed":7,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:57:06.808712+0000","scrub_duration":0.013630486000000001,"objects_trimmed":0,"snaptrim_duration":0.0026815860000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3409,"num_read_kb":8092,"num_write":688,"num_write_kb":6164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":1023710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"181'392","reported_seq":5212,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.799180+0000","last_change":"2024-09-21T15:36:29.799447+0000","last_active":"2024-09-21T15:36:29.799180+0000","last_peered":"2024-09-21T15:36:29.799180+0000","last_clean":"2024-09-21T15:36:29.799180+0000","last_became_active":"2024-09-21T15:32:46.735026+0000","last_became_peered":"2024-09-21T15:32:46.735026+0000","last_unstale":"2024-09-21T15:36:29.799180+0000","last_undegraded":"2024-09-21T15:36:29.799180+0000","last_fullsized":"2024-09-21T15:36:29.799180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'392","last_scrub_stamp":"2024-09-21T15:36:29.799122+0000","last_deep_scrub":"181'392","last_deep_scrub_stamp":"2024-09-21T15:36:29.799122+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.799122+0000","objects_scrubbed":8,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:23.079648+0000","scrub_duration":0.016157649999999999,"objects_trimmed":0,"snaptrim_duration":0.003205977,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4947,"num_read_kb":15952,"num_write":365,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'436","reported_seq":9509,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:39.663815+0000","last_change":"2024-09-21T15:36:39.664094+0000","last_active":"2024-09-21T15:36:39.663815+0000","last_peered":"2024-09-21T15:36:39.663815+0000","last_clean":"2024-09-21T15:36:39.663815+0000","last_became_active":"2024-09-21T15:35:18.853845+0000","last_became_peered":"2024-09-21T15:35:18.853845+0000","last_unstale":"2024-09-21T15:36:39.663815+0000","last_undegraded":"2024-09-21T15:36:39.663815+0000","last_fullsized":"2024-09-21T15:36:39.663815+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'436","last_scrub_stamp":"2024-09-21T15:36:39.663754+0000","last_deep_scrub":"181'436","last_deep_scrub_stamp":"2024-09-21T15:36:39.663754+0000","last_clean_scrub_stamp":"2024-09-21T15:36:39.663754+0000","objects_scrubbed":7,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:48.585537+0000","scrub_duration":0.014179225,"objects_trimmed":0,"snaptrim_duration":0.004536272,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10842,"num_read_kb":21006,"num_write":421,"num_write_kb":11088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":6009675,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"181'316","reported_seq":2326,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:31.661089+0000","last_change":"2024-09-21T15:36:31.661310+0000","last_active":"2024-09-21T15:36:31.661089+0000","last_peered":"2024-09-21T15:36:31.661089+0000","last_clean":"2024-09-21T15:36:31.661089+0000","last_became_active":"2024-09-21T15:32:46.732799+0000","last_became_peered":"2024-09-21T15:32:46.732799+0000","last_unstale":"2024-09-21T15:36:31.661089+0000","last_undegraded":"2024-09-21T15:36:31.661089+0000","last_fullsized":"2024-09-21T15:36:31.661089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'316","last_scrub_stamp":"2024-09-21T15:36:31.661037+0000","last_deep_scrub":"181'316","last_deep_scrub_stamp":"2024-09-21T15:36:31.661037+0000","last_clean_scrub_stamp":"2024-09-21T15:36:31.661037+0000","objects_scrubbed":4,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:10:55.846939+0000","scrub_duration":0.0097720120000000001,"objects_trimmed":0,"snaptrim_duration":0.0029037519999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":1875,"num_read_kb":12552,"num_write":277,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"181'401","reported_seq":17642,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:23.561176+0000","last_change":"2024-09-21T15:36:23.561530+0000","last_active":"2024-09-21T15:36:23.561176+0000","last_peered":"2024-09-21T15:36:23.561176+0000","last_clean":"2024-09-21T15:36:23.561176+0000","last_became_active":"2024-09-21T15:32:46.729638+0000","last_became_peered":"2024-09-21T15:32:46.729638+0000","last_unstale":"2024-09-21T15:36:23.561176+0000","last_undegraded":"2024-09-21T15:36:23.561176+0000","last_fullsized":"2024-09-21T15:36:23.561176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'401","last_scrub_stamp":"2024-09-21T15:36:23.561125+0000","last_deep_scrub":"181'401","last_deep_scrub_stamp":"2024-09-21T15:36:23.561125+0000","last_clean_scrub_stamp":"2024-09-21T15:36:23.561125+0000","objects_scrubbed":6,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:12:01.327173+0000","scrub_duration":0.012493980999999999,"objects_trimmed":0,"snaptrim_duration":0.0057169359999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":18026,"num_read_kb":7731,"num_write":531,"num_write_kb":13565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"181'381","reported_seq":38192,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:32.584334+0000","last_change":"2024-09-21T15:36:32.584580+0000","last_active":"2024-09-21T15:36:32.584334+0000","last_peered":"2024-09-21T15:36:32.584334+0000","last_clean":"2024-09-21T15:36:32.584334+0000","last_became_active":"2024-09-21T15:34:30.765098+0000","last_became_peered":"2024-09-21T15:34:30.765098+0000","last_unstale":"2024-09-21T15:36:32.584334+0000","last_undegraded":"2024-09-21T15:36:32.584334+0000","last_fullsized":"2024-09-21T15:36:32.584334+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'381","last_scrub_stamp":"2024-09-21T15:36:32.584279+0000","last_deep_scrub":"181'381","last_deep_scrub_stamp":"2024-09-21T15:36:32.584279+0000","last_clean_scrub_stamp":"2024-09-21T15:36:32.584279+0000","objects_scrubbed":5,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T20:29:11.771825+0000","scrub_duration":0.0126877,"objects_trimmed":0,"snaptrim_duration":0.005286326,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":38168,"num_read_kb":47931,"num_write":375,"num_write_kb":22505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3360332,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"181'462","reported_seq":2109,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:34.582453+0000","last_change":"2024-09-21T15:36:34.582453+0000","last_active":"2024-09-21T15:36:34.582453+0000","last_peered":"2024-09-21T15:36:34.582453+0000","last_clean":"2024-09-21T15:36:34.582453+0000","last_became_active":"2024-09-21T15:34:33.804147+0000","last_became_peered":"2024-09-21T15:34:33.804147+0000","last_unstale":"2024-09-21T15:36:34.582453+0000","last_undegraded":"2024-09-21T15:36:34.582453+0000","last_fullsized":"2024-09-21T15:36:34.582453+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'462","last_scrub_stamp":"2024-09-21T15:36:34.582159+0000","last_deep_scrub":"181'462","last_deep_scrub_stamp":"2024-09-21T15:36:34.582159+0000","last_clean_scrub_stamp":"2024-09-21T15:36:34.582159+0000","objects_scrubbed":7,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:56:38.352489+0000","scrub_duration":0.015072914999999999,"objects_trimmed":0,"snaptrim_duration":0.0054559919999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2854,"num_read_kb":9000,"num_write":510,"num_write_kb":5133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2531603,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"181'314","reported_seq":1530,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:30.800025+0000","last_change":"2024-09-21T15:36:30.800249+0000","last_active":"2024-09-21T15:36:30.800025+0000","last_peered":"2024-09-21T15:36:30.800025+0000","last_clean":"2024-09-21T15:36:30.800025+0000","last_became_active":"2024-09-21T15:35:54.783023+0000","last_became_peered":"2024-09-21T15:35:54.783023+0000","last_unstale":"2024-09-21T15:36:30.800025+0000","last_undegraded":"2024-09-21T15:36:30.800025+0000","last_fullsized":"2024-09-21T15:36:30.800025+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'314","last_scrub_stamp":"2024-09-21T15:36:30.799969+0000","last_deep_scrub":"181'314","last_deep_scrub_stamp":"2024-09-21T15:36:30.799969+0000","last_clean_scrub_stamp":"2024-09-21T15:36:30.799969+0000","objects_scrubbed":12,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:40.892369+0000","scrub_duration":0.018160714000000001,"objects_trimmed":0,"snaptrim_duration":0.00095779800000000005,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":1163,"num_read_kb":7872,"num_write":259,"num_write_kb":6145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":6804015,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"181'623","reported_seq":6853,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:22.536127+0000","last_change":"2024-09-21T15:36:22.536334+0000","last_active":"2024-09-21T15:36:22.536127+0000","last_peered":"2024-09-21T15:36:22.536127+0000","last_clean":"2024-09-21T15:36:22.536127+0000","last_became_active":"2024-09-21T15:32:46.731820+0000","last_became_peered":"2024-09-21T15:32:46.731820+0000","last_unstale":"2024-09-21T15:36:22.536127+0000","last_undegraded":"2024-09-21T15:36:22.536127+0000","last_fullsized":"2024-09-21T15:36:22.536127+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'623","last_scrub_stamp":"2024-09-21T15:36:22.536072+0000","last_deep_scrub":"181'623","last_deep_scrub_stamp":"2024-09-21T15:36:22.536072+0000","last_clean_scrub_stamp":"2024-09-21T15:36:22.536072+0000","objects_scrubbed":4,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:28:55.126061+0000","scrub_duration":0.011084538,"objects_trimmed":0,"snaptrim_duration":0.0040997610000000004,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9129,"num_read_kb":23531,"num_write":702,"num_write_kb":19195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"181'616","reported_seq":7864,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:29.905493+0000","last_change":"2024-09-21T15:36:29.905738+0000","last_active":"2024-09-21T15:36:29.905493+0000","last_peered":"2024-09-21T15:36:29.905493+0000","last_clean":"2024-09-21T15:36:29.905493+0000","last_became_active":"2024-09-21T15:35:54.787545+0000","last_became_peered":"2024-09-21T15:35:54.787545+0000","last_unstale":"2024-09-21T15:36:29.905493+0000","last_undegraded":"2024-09-21T15:36:29.905493+0000","last_fullsized":"2024-09-21T15:36:29.905493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'616","last_scrub_stamp":"2024-09-21T15:36:29.905433+0000","last_deep_scrub":"181'616","last_deep_scrub_stamp":"2024-09-21T15:36:29.905433+0000","last_clean_scrub_stamp":"2024-09-21T15:36:29.905433+0000","objects_scrubbed":4,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:20:10.182957+0000","scrub_duration":0.010341866999999999,"objects_trimmed":0,"snaptrim_duration":0.00154619,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7391,"num_read_kb":18764,"num_write":724,"num_write_kb":22843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":2952900,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"181'493","reported_seq":11007,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-21T15:36:24.575997+0000","last_change":"2024-09-21T15:36:24.576220+0000","last_active":"2024-09-21T15:36:24.575997+0000","last_peered":"2024-09-21T15:36:24.575997+0000","last_clean":"2024-09-21T15:36:24.575997+0000","last_became_active":"2024-09-21T15:32:46.737371+0000","last_became_peered":"2024-09-21T15:32:46.737371+0000","last_unstale":"2024-09-21T15:36:24.575997+0000","last_undegraded":"2024-09-21T15:36:24.575997+0000","last_fullsized":"2024-09-21T15:36:24.575997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'493","last_scrub_stamp":"2024-09-21T15:36:24.575939+0000","last_deep_scrub":"181'493","last_deep_scrub_stamp":"2024-09-21T15:36:24.575939+0000","last_clean_scrub_stamp":"2024-09-21T15:36:24.575939+0000","objects_scrubbed":5,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:14.282186+0000","scrub_duration":0.011960442999999999,"objects_trimmed":0,"snaptrim_duration":0.0057583979999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":13492,"num_read_kb":33833,"num_write":422,"num_write_kb":15145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":253,"num_read":245529,"num_read_kb":504449,"num_write":17749,"num_write_kb":381766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":43724062,"num_keys_recovered":230,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},"log_size":15610,"ondisk_log_size":15610,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42016,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329824,"statfs":{"total":96636764160,"available":96593739776,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.878},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89700000000000002,"15min":0.82999999999999996},"min":{"1min":0.49199999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.1830000000000001,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.81699999999999995,"15min":0.85699999999999998},"min":{"1min":0.53100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.3029999999999999,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.79900000000000004,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.077,"5min":1.252,"15min":1.252},"last":1.006},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.621,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.252,"5min":3.1709999999999998,"15min":3.1709999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1459999999999999,"5min":9.8130000000000006,"15min":9.8130000000000006},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.83099999999999996,"15min":0.873},"min":{"1min":0.54300000000000004,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0660000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.81299999999999994,"15min":0.86299999999999999},"min":{"1min":0.48799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1899999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.087},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.95299999999999996,"15min":0.879},"min":{"1min":0.45600000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2370000000000001,"5min":9.7490000000000006,"15min":9.7490000000000006},"last":0.97299999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.89800000000000002,"15min":0.94399999999999995},"min":{"1min":0.436,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.3360000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.048,"5min":0.91400000000000003,"15min":0.92800000000000005},"min":{"1min":0.50900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.91000000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":0.95899999999999996,"15min":1.004},"min":{"1min":0.53100000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.296,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.073,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.276,"5min":9.6799999999999997,"15min":9.6799999999999997},"last":1.0309999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45532,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94326308,"statfs":{"total":96636764160,"available":96590139392,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.1439999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2330000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.82499999999999996,"15min":0.86699999999999999},"min":{"1min":0.626,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.218,"5min":1.452,"15min":1.452},"last":0.96699999999999997}]},{"osd":2,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.59699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3819999999999999,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.53600000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.107,"5min":2.423,"15min":2.423},"last":1.0089999999999999}]},{"osd":3,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.86899999999999999,"15min":0.878},"min":{"1min":0.54800000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.2929999999999999,"5min":1.554,"15min":1.554},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.627,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.357,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":0.92900000000000005}]},{"osd":4,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88200000000000001,"15min":0.90400000000000003},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.373,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.331,"5min":5.6130000000000004,"15min":5.6130000000000004},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91300000000000003,"15min":0.93200000000000005},"min":{"1min":0.46800000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.276,"5min":3.5499999999999998,"15min":3.5499999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.99399999999999999,"15min":0.995},"min":{"1min":0.38700000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.417,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.88600000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0249999999999999,"15min":1.014},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3460000000000001,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.004,"15min":1.0529999999999999},"min":{"1min":0.57099999999999995,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4239999999999999,"5min":3.4740000000000002,"15min":3.4740000000000002},"last":1.048}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.054},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.86699999999999999}]},{"osd":1,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.77600000000000002,"15min":0.76700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4319999999999999,"5min":1.8,"15min":1.8},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.77800000000000002,"15min":0.77100000000000002},"min":{"1min":0.51000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.296,"5min":1.323,"15min":1.323},"last":1.0960000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83899999999999997,"15min":0.79000000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.196,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.82199999999999995,"15min":0.81200000000000006},"min":{"1min":0.47399999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4099999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.81799999999999995}]},{"osd":3,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.621,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.26,"5min":4.1989999999999998,"15min":4.1989999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91800000000000004,"15min":0.92000000000000004},"min":{"1min":0.54900000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2769999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.97499999999999998}]},{"osd":4,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2070000000000001,"5min":1.476,"15min":1.476},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.94999999999999996,"15min":0.92400000000000004},"min":{"1min":0.52500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.327,"5min":4.4809999999999999,"15min":4.4809999999999999},"last":1.1399999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92900000000000005,"15min":0.91500000000000004},"min":{"1min":0.39100000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2869999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92300000000000004,"15min":0.93999999999999995},"min":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.373,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.012,"15min":0.99399999999999999},"min":{"1min":0.439,"5min":0.432,"15min":0.432},"max":{"1min":1.5069999999999999,"5min":2.214,"15min":2.214},"last":1.014},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.1140000000000001,"15min":1.052},"min":{"1min":0.56200000000000006,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3839999999999999,"5min":4.5730000000000004,"15min":4.5730000000000004},"last":1.026}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41440,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330400,"statfs":{"total":96636764160,"available":96594329600,"internally_reserved":0,"allocated":2187264,"data_stored":1050819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.88800000000000001}]},{"osd":1,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.80400000000000005},"min":{"1min":0.52600000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0960000000000001,"5min":3.069,"15min":3.069},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.77900000000000003,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.153,"5min":2.952,"15min":2.952},"last":0.81599999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.59999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2030000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.876,"15min":0.94199999999999995},"min":{"1min":0.495,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1180000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.077}]},{"osd":3,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.84299999999999997},"min":{"1min":0.48299999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.129,"5min":2.048,"15min":2.048},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85799999999999998,"15min":0.88900000000000001},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1990000000000001,"5min":3.218,"15min":3.218},"last":1.101}]},{"osd":5,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.81699999999999995,"15min":0.873},"min":{"1min":0.53100000000000003,"5min":0.313,"15min":0.313},"max":{"1min":1.1299999999999999,"5min":3.0289999999999999,"15min":3.0289999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.86299999999999999,"15min":0.877},"min":{"1min":0.40500000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2330000000000001,"5min":6.3369999999999997,"15min":6.3369999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.87,"15min":0.88},"min":{"1min":0.57299999999999995,"5min":0.31,"15min":0.31},"max":{"1min":1.159,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":1.149},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.96499999999999997,"15min":0.995},"min":{"1min":0.67300000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.232,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":1.1259999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.87,"15min":0.872},"min":{"1min":0.47899999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.298,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.98899999999999999,"15min":1.016},"min":{"1min":0.61199999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2450000000000001,"5min":3.2839999999999998,"15min":3.2839999999999998},"last":0.72199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55388,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316452,"statfs":{"total":96636764160,"available":96580046848,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56999999999999995}]},{"osd":1,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.61899999999999999},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.4339999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61799999999999999,"15min":0.625},"min":{"1min":0.31900000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.034,"5min":1.127,"15min":1.127},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.68899999999999995},"min":{"1min":0.33900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.042,"5min":1.802,"15min":1.802},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.71799999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2230000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.91300000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.78900000000000003,"15min":0.80700000000000005},"min":{"1min":0.57199999999999995,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.262,"5min":1.587,"15min":1.587},"last":0.871},{"interface":"front","average":{"1min":0.85099999999999998,"5min":1.135,"15min":1.0149999999999999},"min":{"1min":0.60899999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.143,"5min":24.064,"15min":24.064},"last":0.83999999999999997}]},{"osd":5,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.84699999999999998,"15min":0.91300000000000003},"min":{"1min":0.60599999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.337,"5min":1.966,"15min":1.966},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.91500000000000004},"min":{"1min":0.58999999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.329,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.98999999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86799999999999999,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.272,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.997,"5min":0.91800000000000004,"15min":0.96399999999999997},"min":{"1min":0.64600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5760000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Sep 21 15:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.93899999999999995,"15min":0.97699999999999998},"min":{"1min":0.54400000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.4359999999999999,"5min":1.772,"15min":1.772},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93100000000000005,"15min":1.004},"min":{"1min":0.63700000000000001,"5min":0.498,"15min":0.498},"max":{"1min":1.397,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41436,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330404,"statfs":{"total":96636764160,"available":96594333696,"internally_reserved":0,"allocated":2183168,"data_stored":1050800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68300000000000005}]},{"osd":1,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69099999999999995,"15min":0.745},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93700000000000006,"5min":1.613,"15min":1.613},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.752},"min":{"1min":0.35699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0660000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93400000000000005}]},{"osd":3,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.33500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66800000000000004,"15min":0.69599999999999995},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.004,"5min":1.159,"15min":1.159},"last":0.98499999999999999}]},{"osd":4,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.86699999999999999,"15min":0.90700000000000003},"min":{"1min":0.64900000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2270000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.89500000000000002,"15min":0.91600000000000004},"min":{"1min":0.66000000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2170000000000001,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73199999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.48799999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.339,"5min":1.536,"15min":1.536},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.54200000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2849999999999999,"5min":1.52,"15min":1.52},"last":0.90400000000000003}]},{"osd":6,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.90700000000000003,"15min":0.95799999999999996},"min":{"1min":0.59599999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.131,"5min":1.617,"15min":1.617},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.92900000000000005,"15min":0.97099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.252,"5min":1.554,"15min":1.554},"last":0.86899999999999999}]},{"osd":7,"last update":"Sat Sep 21 15:36:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.95799999999999996,"15min":0.99199999999999999},"min":{"1min":0.58399999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.387,"5min":1.669,"15min":1.669},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.085,"5min":0.96399999999999997,"15min":0.99299999999999999},"min":{"1min":0.62,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4590000000000001,"5min":1.589,"15min":1.589},"last":1.097}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2191360,"data_stored":1050838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 21 15:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60099999999999998,"15min":0.57599999999999996},"min":{"1min":0.36899999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93899999999999995,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63600000000000001,"15min":0.60399999999999998},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.83899999999999997}]},{"osd":3,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.38400000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.32700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.99199999999999999,"5min":1.323,"15min":1.323},"last":0.90600000000000003}]},{"osd":4,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83099999999999996,"15min":0.79900000000000004},"min":{"1min":0.56799999999999995,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.081,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.64000000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.083}]},{"osd":5,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.62,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3200000000000001,"5min":1.514,"15min":1.514},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.91000000000000003,"15min":0.876},"min":{"1min":0.51400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3069999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.88600000000000001}]},{"osd":6,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":0.95299999999999996,"15min":0.96199999999999997},"min":{"1min":0.65800000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.651,"5min":1.891,"15min":1.891},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.036,"5min":0.94499999999999995,"15min":0.94699999999999995},"min":{"1min":0.53500000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.407,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.2250000000000001}]},{"osd":7,"last update":"Sat Sep 21 15:36:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.98299999999999998,"15min":1.002},"min":{"1min":0.64000000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.48,"5min":1.502,"15min":1.502},"last":1.274},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.97099999999999997,"15min":0.94499999999999995},"min":{"1min":0.55200000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3200000000000001,"5min":3.8359999999999999,"15min":3.8359999999999999},"last":1.133}]}]},{"osd":0,"up_from":162,"seq":695784701969,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34528,"kb_used_data":2712,"kb_used_omap":15,"kb_used_meta":31792,"kb_avail":94337312,"statfs":{"total":96636764160,"available":96601407488,"internally_reserved":0,"allocated":2777088,"data_stored":1641168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15601,"internal_metadata":32555791},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63600000000000001}]},{"osd":4,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.97799999999999998}]},{"osd":5,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":1.002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.0289999999999999}]},{"osd":6,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.091}]},{"osd":7,"last update":"Sat Sep 21 15:36:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.0669999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6096,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T15:36:57.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T15:36:57.610 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T15:36:57.610 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T15:36:57.611 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T15:36:57.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:36:57.685 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T15:36:57.685 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T15:36:57.685 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:36:57.763 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T15:36:57.763 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T15:36:57.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:00.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-21T15:37:03.864 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T15:37:03.865 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T15:37:03.865 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:03.924 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T15:37:03.924 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T15:37:03.924 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:04.013 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T15:37:04.013 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T15:37:04.013 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:04.091 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T15:37:04.091 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T15:37:04.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:05.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T15:37:05.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-21T15:37:10.193 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T15:37:10.193 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T15:37:10.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.258 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T15:37:10.259 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T15:37:10.259 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T15:37:10.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.286 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T15:37:10.286 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T15:37:10.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.316 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T15:37:10.316 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T15:37:10.317 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T15:37:10.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.339 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T15:37:10.340 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T15:37:10.340 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.396 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T15:37:10.396 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T15:37:10.396 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T15:37:10.416 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T15:37:10.417 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T15:37:10.417 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-21T15:37:10.453 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:37:10.453 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T15:37:10.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T15:37:10.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:37:10.647 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T15:37:10.840 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:37:10.840 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T15:37:10.971 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:37:10.971 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T15:37:11.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:37:11.133 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T15:37:11.395 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:37:11.395 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T15:37:11.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:37:11.664 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T15:37:11.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:37:11.847 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T15:37:12.065 INFO:tasks.ceph:Archiving mon data... 2024-09-21T15:37:12.066 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/data/mon.a.tgz 2024-09-21T15:37:12.066 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-21T15:37:12.072 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.YEbvrM9bG5 2024-09-21T15:37:12.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.YEbvrM9bG5 2024-09-21T15:37:12.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.YEbvrM9bG5 2024-09-21T15:37:12.282 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.YEbvrM9bG5 is 429KB 2024-09-21T15:37:12.345 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.YEbvrM9bG5 2024-09-21T15:37:12.350 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/data/mon.c.tgz 2024-09-21T15:37:12.350 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-21T15:37:12.395 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.RhFdKFYZIo 2024-09-21T15:37:12.395 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.RhFdKFYZIo 2024-09-21T15:37:12.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.RhFdKFYZIo 2024-09-21T15:37:12.642 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.RhFdKFYZIo is 585KB 2024-09-21T15:37:12.707 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.RhFdKFYZIo 2024-09-21T15:37:12.712 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/data/mon.b.tgz 2024-09-21T15:37:12.713 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-09-21T15:37:12.718 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.EtCb2aIOgO 2024-09-21T15:37:12.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EtCb2aIOgO 2024-09-21T15:37:12.905 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.EtCb2aIOgO 2024-09-21T15:37:12.969 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.EtCb2aIOgO is 585KB 2024-09-21T15:37:13.031 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.EtCb2aIOgO 2024-09-21T15:37:13.036 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T15:37:13.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T15:37:13.039 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T15:37:13.094 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-21T15:37:13.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T15:37:13.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-21T15:37:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T15:37:13.160 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-21T15:37:13.163 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T15:37:13.165 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi107/crash 2024-09-21T15:37:13.166 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T15:37:13.180 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi170/crash 2024-09-21T15:37:13.181 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T15:37:13.217 INFO:tasks.ceph:Compressing logs... 2024-09-21T15:37:13.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:37:13.228 DEBUG:teuthology.orchestra.run.smithi170:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:37:27.131 INFO:tasks.ceph:Archiving logs... 2024-09-21T15:37:27.131 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi107/log 2024-09-21T15:37:27.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T15:37:28.266 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi170/log 2024-09-21T15:37:28.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T15:37:29.305 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T15:37:29.314 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T15:37:29.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T15:37:29.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T15:37:29.395 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T15:37:29.403 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T15:37:29.403 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:37:29.406 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T15:37:29.417 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:37:29.417 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-09-21T15:37:29.417 INFO:teuthology.orchestra.run.smithi170.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 177 0.089 -0.988 0.822 2024-09-21T15:37:29.417 INFO:teuthology.orchestra.run.smithi170.stdout:+hv02.front.sepi 96.245.170.99 3 u 45 64 177 0.125 +0.487 0.881 2024-09-21T15:37:29.417 INFO:teuthology.orchestra.run.smithi170.stdout:*hv03.front.sepi 74.6.168.72 3 u 51 64 177 0.140 -1.930 0.931 2024-09-21T15:37:29.418 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:37:29.418 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2024-09-21T15:37:29.419 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2024-09-21T15:37:29.419 INFO:teuthology.orchestra.run.smithi107.stdout:*hv01.front.sepi 67.205.162.81 3 u 45 64 177 0.054 -0.881 0.899 2024-09-21T15:37:29.419 INFO:teuthology.orchestra.run.smithi107.stdout:+hv02.front.sepi 96.245.170.99 3 u 45 64 177 0.082 +0.186 1.032 2024-09-21T15:37:29.419 INFO:teuthology.orchestra.run.smithi107.stdout:+hv03.front.sepi 74.6.168.72 3 u 50 64 177 0.085 -1.023 0.913 2024-09-21T15:37:29.419 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-21T15:37:29.420 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T15:37:29.428 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T15:37:29.428 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T15:37:29.436 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T15:37:29.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T15:37:29.453 INFO:teuthology.task.internal:Duration was 898.615892 seconds 2024-09-21T15:37:29.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T15:37:29.461 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T15:37:29.462 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T15:37:29.464 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T15:37:29.500 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T15:37:29.500 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T15:37:29.501 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T15:37:29.552 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-09-21T15:37:29.552 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T15:37:29.562 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T15:37:29.562 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:37:29.596 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T15:37:29.633 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T15:37:29.633 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T15:37:29.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T15:37:29.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T15:37:29.755 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T15:37:29.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T15:37:29.767 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T15:37:29.777 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T15:37:29.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:37:29.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:37:29.820 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-21T15:37:29.831 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-09-21T15:37:29.926 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:37:29.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:37:29.932 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T15:37:29.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T15:37:29.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T15:37:29.982 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T15:37:29.983 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi107 2024-09-21T15:37:29.983 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T15:37:30.008 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909985/remote/smithi170 2024-09-21T15:37:30.009 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T15:37:30.036 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T15:37:30.036 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T15:37:30.048 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T15:37:30.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T15:37:30.092 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T15:37:30.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T15:37:30.103 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T15:37:30.103 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T15:37:30.106 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T15:37:30.109 INFO:teuthology.orchestra.run.smithi107.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 15:37 /home/ubuntu/cephtest 2024-09-21T15:37:30.130 INFO:teuthology.orchestra.run.smithi170.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 15:37 /home/ubuntu/cephtest 2024-09-21T15:37:30.131 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T15:37:30.140 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T15:37:30.161 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 898.6158916950226 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T15:37:30.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T15:37:30.248 INFO:teuthology.run:pass